Jeremy’s Quest to Harness the Power of Search – The Beginning

I’ve finally taken the time to notice that ever since I lost all the ramble from the past and had to start over again, search traffic had become pretty much non existant. The only searches that are still coming through are all somehow related to shrunken men and from Yahoo. If you don’t know the full story about the shrunken men yet, don’t ask… just search. 🙂

Anyhow. I decided that I should do my part to make sure that everything is optimized and ready for the crawlers to come through and get it. I also need to make sure that the crawlers know what they’re looking for. In order to half ass some kind of structure to this process, I’m going to base what the search engines know about me on a post that I made last week regarding the stealing of books. That post has been bumped off the first page, so the only way a search engine should now see it is through that page, the category, and directly to the permanent link itself. I’m going to do my best not to mention the actual title of the post so as not to skew the results, but if you’re that interested, do a site search in the top right for books and you can figure it out.

The good stuff is after the jump

Step 1: Google

Google is funny. If I search for the title of said post I get 3 results, none of them linking to ramblewords.com at all. All 3 links are to other blog search/feed services that have successfully indexed the feed. Google only allows me to go to those sites to read my content. Awesome.

Google should also be easy to fix. I had forgotten that in the past I made a sitemap.xml file for their sitemaps program. If you aren’t familiar, it basically provides an exact structure of your site so that the googlebot knows what it’s looking for.

Today I updated that and made sure that Google was aware of the new location. The bot should now stop paying attention to the old structure and have a go at the new one. I’m expecting the results to get better in the next day or so.

Step 2: Yahoo

Yahoo is dumb. That’s the only conclusion that I can come to. Right now when I search for said title, I get 4 results. 3 of these results are to spam blogs, which don’t even provide a clear link to the post in question. 1 of the results is straight to the main page at this blog, but the post has been bumped off the main page so it is no longer relevant.

I’m not exactly sure what I have to do to fix this one. Yahoo’s bot seems to be caught up on my old site structure, but there is not fancy sitemap that I can submit to them. I did go to their “submit my site” section and resubmit, so hopefully it will cause a reindexing to occur. I’m not holding out too much hope though, so I’ll research a few other methods and see what happens.

Step 3: Ask.com

I’ll admit that I don’t even use ask.com, so I’m not even sure if I used to show up there. Right now though, I get 0 results for a web search with the title of the post. The site that I found to submit the site to Ask was down, so I’m going to keep trying later until I’m able to add it. I know I used to get a few clicks from Ask related to image searches, but that’s gone way down.

Step 4: Microsoft Live.com

This one is amusing too. 2 results show up out of 3 found. Neither of the results point directly to the page in question or to the hope page, but point to the 2 categories that the post is made in. This works for now, but I’m still unsure as to why the actual post itself is not linked. I’m still exploring the Live ways of search, so hopefully I’ll find a good way to get the bot looking at the right stuff soon.

Step 5: MSN Search

That brings us to our next little buddy. Proving once and for all that Jeremy can’t stand Microsoft search tools… ha ha. Personal thoughts aside, msn.com returned 1 result showing 1 category link. That’s it. It didn’t even catch the other category that the post was made in, no link to the main page, no link directly to the post. I’m hoping that by playing around with some of the MSN.com stuff, the live.com will be updated as well. Still gotta find how to do that though.

Side Mention: Blog searches:

The good part about all of this is that the most relevant places did have the page indexed properly. Technorati has a link directly to the correct post as the only result for the blog search. Google’s blogsearch also has a link directly to the correct post as the only result. I still have plenty more to check, but those are the most important right now.

So now that my whining is done, I will leave this alone for 2 days and come back to check on the progress. That should be plenty of internet time for the search engines to pull their heads out of the sand and start checking the new stuff that I’m sending them. I’m mainly interested in Google and Yahoo getting it right, but it’d be fun to see everything else populate as well.