So I crank back in front of the computer after pigging out at the Thanksgiving table, and I’m just sort of enjoying the fact this site is coming together.  It no longer looks like ass, I’ve successfully moved it over to pMachine hosting, I’ve started getting up some nice new downloads and added a few useful sections to the site, site membership keeps climbing and I’m finally giving ya’ll something to look at.  Joy!

And then I notice it, lurking on my toolbar.  Taunting me with its lack of green goodness.  My page rank- or lack thereof.  Yes, indeedy, I have PageRank of 0.  That’s 0 as in ‘ZERO’.  Jeesh, I haven’t had a site with 0 PageRank since I knew what it was!  To make things worse- looks like all of my pages aside from the index are banished to Google’s ‘Supplemental results’- i.e., the kiss of death.

What to do, what to do?  Get to work fixing it- and document the process in a series of posts.  This?  Would be the first.

Want to know about what to do- and what NOT to do- in order to make an ExpressionEngine powered site search engine friendly?  Read on.  In this entry, I’ll try and track down the problem areas so I can start to fix them.  Where to start?  Google, Yahoo and sitemaps.

Read on....

Now I know ‘PageRank’ isn’t the ‘end all, be all’, and I know that the only thing that directly impacts a site’s PageRank is the number and quality of inbound links.  There’s still a lot you can do to make it easier for bots and humans to find you. 

To start my SEO odyssey, I wanted to do two things.
1.  Identify the pages I want the spiders to find and make sure they can find them!  To help me do this, I added a regular html sitemap and included a link to it in my footer.  I also created an xml sitemap and submitted it to Yahoo and Google.  Then I wrote a little extension, making it easy to ping said search engines when I add a new entry, thus making sure Google and Yahoo always have a fresh copy.

2.  Identify the pages the spiders ARE finding and make sure they’re the right ones.  To do this, I got a Yahoo ID and checked out Yahoo’s Site Explorer; I used my existing Google account to check out its Webmaster tools.  What I found told me what I need to do next.

Want some details?  You got ‘em.

Adding an (x)html site map
Creating a dynamic html sitemap for your visitors takes all of 5 minutes with EE.  And if you are wise and use an embed to hold a header or footer or nav bar where you can add your link to your new sitemap- you can easily tie it into all of your pages in another minute.  Have a huge site?  Break your sitemap up into pages.  Whatever you do, be sure to make wise use of the disable parameter to keep your queries tight.  Here’s my very simple html sitemap template.

Adding an xml sitemap
I decided that while I was at it, I’d also add a ‘Google sitemap’- an XML list of site urls that’s meant to help webmasters get better, more efficient crawls from spiders.  Happily, Yahoo and MSN have joined in supporting the protocol (see sitemaps.org).  Even more happily, with just a little tweaking of the EE Wiki tutorial, it was a matter of minutes to have a working xml sitemap.

The code I’m using requires a bit of php, as I wanted my modified date to reflect the latest comments where appropriate.  Also, given the small size of my site, it seemed pointless to add priority tags.  And from a bit of reading, I figured having both a changefreq tag and a lastmodified was redundant.  It’s a dynamic site- let’s go with last modified.  Once I get a couple of hundred posts, I may add in priority- if so, I’ll do it dynamically, giving the highest priority to the newest entries.  Gotta love php/mysql!

If you’re rolling your own xml sitemap, I’d suggest using the Wiki as a guide.  If you’re curious as to how I went at it- the basic gist is: set template type to xml, turn php ‘on’ for the template, parsed on output, and do something like: my xml sitemap template - which produces this xml.

The main bit where I bring in some php is to show the lastmod on the single entry pages based on either the last comment date or the last edit date- whichever is newer.  The core php looks like

<?php
$c_date = '{recent_comment_date}';
$e_date = $LOC->timestamp_to_gmt('{edit_date}');
$updated = ($c_date > $e_date) ? $LOC->decode_date($LOC->format['DATE_W3C'], $c_date) : $LOC->decode_date($LOC->format['DATE_W3C'], $e_date);
?>

Then I just echo the $updated variable back as the lastmod time.  Note- this is completely not necessary, I just enjoyed dinking with it.  If the php freaks you out- just copy the wiki.

The sitemaps are good to go- and unless I add more sections, they’ll do a nice job of reflecting the site’s content without any tweaking from me.  I even wrote up a handy Google, Yahoo Sitemap ping extension that automates sending Google and Yahoo update notification.  Basically- I’m gearing this site up to support my complete and total laziness.

Getting a Spider-eye view
Now that I’ve created my sitemaps, I’ve got a darn good idea of the pages I want picked up by the search bots.  It’s time to take a look at what the bots DO see.

Google- you probably already have a google account- at least, I did.  If not, there’s a sign-up link on the Google Webmaster tools page

While I was there, I added my sitemap, verified my ownership of the site, and selected a preference for references to my site not to use the www.  Then I took a look around.  The most relevant page at this point in time is a simple search of my site: site:media-cow.com.  No- you really don’t need the webmaster tools to do that!  What did I find?  At the time, there were 130+ indexed links- of which exactly 1 (the homepage) was not in supplemental results.  I took a screen shot today (there’s already a bit of improvement)- if you look closely, you’ll be able to guess what the topic of the next installment will be.
image

Yahoo doesn’t keep a supplemental index (yay)- but they did show me with 209 pages indexed.  Now honestly?  I’ve got around 30 pages of content on this site- and that’s stretching it.  That many pages being indexed suggests something is up- and a look at the first page of the Yahoo index gives me a good idea of what.
image

Lastly- just for fun- I wanted a quick and easy way to keep track of the effects of my SEO efforts- so I hopped over to Zippy and took a screenshot of the info on my domain.  It’s a quick and dirty way to see how my tweaks work across Google, Yahoo, MSN and Alexa.
image

Summing up
So- what have I accomplished so far?  I figured out my site was hosed in the search engines!  The search engine indexes are full of crap pages, non-existent pages, duplicate content.  My tiny little island of REAL content is buried in crap.  In truth, sitemaps aren’t going to do much to fix the problems on this site and they sure won’t improve my PageRank.  So for the next installment, I’ll talk about how I’m weeding out the filler- and how a little thought and planning will help prevent it from happening in the first place.