Results tagged “web”

Drupal to Movable Type

Now, I’ve done it. I’ve relaunched my blog using a design I’ve been building on for a few months. (I don’t have much time for such things, so it takes some time to build anything out for this site.) Anyway, there’s more bits and bobbles to tweak out, but the basics are now there. It’s quite an update since I haven’t really dinked with either the overall design or platform running my web site for a few years. The big technical change has been that I am now using Movable Type rather than Drupal to serve my blogs. I’m going to start by explaining why and then I’ll delve into how I made the move.

Why Movable Type?

The main reason for the switch from Drupal to Movable Type is because I wanted to try something new. I’ve been on Drupal for a long time and am familiar with it. I’m also likely to maintain that familiarity for quite sometime since we use Drupal extensively at work. To paraphrase The Matrix Revolutions, “You do not truly know an application until you fight it.” In this case, I’m running Movable Type and getting a feel and we’ll see how long it lasts. In the past, I’ve had similar short runs of Blosxom, WordPress, and a few experiments in creating my very own brand of blogging software for kicks and giggles. Drupal has outlasted them all. If I decided I don’t like this experiment, I may go back.

Movable Type is written primarily in Perl. I like Perl and think it beats the pants off of PHP as far as languages go. Drupal is written in PHP. I have written some plugins for Drupal in PHP, but I’d rather be writing Perl. So, if I’m going to dink around with something at home for fun, I figure it ought to be in something I enjoy.

By using Movable Type, I have been able to vastly improve the performance of my blogs. I host on Dreamhost, which is great for the price, but performance is an issue from time to time since it is a shared hosting solution. Movable Type publishes most of my pages as static HTML (the ultimate page caching system!) and then updates them as needed when new posts or comments are added. My web site is much zippier now since it doesn’t have to what for PHP (or even Perl CGI/FastCGI) to load for every page.

Movable Type does blogging with a lot of subtlety and nuance. If you look at how everything works, there are a lot of teeny details that it does just right. Drupal does general content management pretty well, but blogging is done just alright. For example, Movable Type provides Atom and RSS feeds out of the box, Drupal provides just RSS unless you add a module on. Movable Type provides RDF and other summaries as part of every post to make it easier for bots to understand my site. The widgets (similar to Drupal’s blocks) are geared towards blogging and seem to make more sense overall. It’s a tool tailored to the specific job rather than a tool tailored to a larger job. I’m not at all knocking Drupal. But even a really great Swiss Army knife isn’t the best screwdriver, it just has a pretty decent one available.

Those are the main reasons for moving to Movable Type. Now, on to how to do it.

Migration from Drupal to Movable Type

Most of the information I was able to find on moving away from one platform to another was, unsurprisingly, Movable Type to Drupal. I found some folks looking or information on going the other way, but nothing definite. I’m going to explain the process I used in hopes that someone else might find it useful if they decide to make a similar move.

Database Conversion

The first and most complicated problem is moving the data. Movable Type helped to simplify this process by providing a standardized import tool. If you go into the back-end MT interface and look on the home page, there’s a link in the right sidebar labeled “Import Content”. If you follow that you can see a form that lets you import data in Movable Type import format or WordPress Import Format. Since I couldn’t find any information on the WordPress format and since I didn’t use WordPress, I left that alone.

The documentation for the Movable Type format is readily available here. That’s not actually the whole story, though. I discovered, by experimenting with the data export function, that there are some additional fields, such as “TAGS,” which allow you to import tags and such as well. Getting the Drupal data out was relatively easy since this is a very simple text format.

I have a little trick I do whenever I need to perform a one-time script on a Drupal instance. The trick is that I write a PHP script that does whatever I need, go into Drupal to create a new page, paste the code into that page, and then switch the input format to PHP. Then, instead of creating the page, I click “Preview,” which executes the script and does whatever I want. I wrote such a script for this occasion.

Here it is:

<?php
header('Content-type: text/plain');
$nids = db_query("select distinct n.nid from {node} n inner join {term_node} t on t.nid = n.nid where t.tid = 360 or t.tid = 362");
while ($node = node_load(db_fetch_object($nids))) {
  print "AUTHOR: ".($node->name=="Andrew Sterling Hanenkamp"?"zostay":$node->name)."\n";
  print "TITLE: ".$node->title."\n";
  print "STATUS: ".($node->status?"Publish":"Draft")."\n";
  print "ALLOW COMMENTS: ".($node->comment>0)."\n";
  print "CONVERT BREAKS: 1\n";
  print "ALLOW PINGS: ".($node->comment>0)."\n";
  print "DATE: ".strftime("%m/%d/%Y %I:%M:%S %p", $node->created)."\n";
  $tag = array();
  foreach ($node->category as $category) {
    $tag[] = $category->title;
  }
  print "TAGS: ".implode(",", $tag)."\n";
  print "-----\n";
  print "BODY:\n";
  print $node->body."\n";
  print "-----\n";
  print "KEYWORDS:\n";
  print "/node/".$node->nid.",".url('node/'.$node->nid)."\n";
  $comments = db_query("select * from {comments} c where c.nid = %d", $node->nid);
  while ($comment = db_fetch_object($comments)) {
    print "-----\n";
    print "COMMENT:\n";
    print "AUTHOR: ".$comment->name."\n";
    print "EMAIL: ".$comment->mail."\n";
    print "IP: ".$comment->hostname."\n";
    print "URL: ".$comment->homepage."\n";
    print "DATE: ".strftime("%m/%d/%Y %I:%M:%S %p", $comment->timestamp)."\n";
    print "SUBJECT: ".$comment->subject."\n";
    print $comment->comment."\n";
  }
  print "--------\n";
}
exit;
?>

This script takes all the nodes belonging to the categories with IDs 360 or 362 and outputs the files in the Movable Type export format. If you change the first query to use a different set of terms or remove them you can export everything or some different subset. I did this because I had 4 web sites served from one Drupal database and the terms determined which site they appeared on. I then condensed two sites together, which is which site this is.

The export script applies all terms as the tags. The script could be customized to narrow that down for your site, if needed.

The script had two basic flaws, which I fixed after the export and reimport (rather than doing everything over again). First, the “SUBJECT:” line from the comments didn’t look as good as I wanted. I’d recommend changing that to “Subject:” or removing it all together. I used the Search and Replace feature of Movable Type to correct it. The script also added “————” at the end of some of the comments, which I also removed using Search and Replace.

The other nice thing I did here (at least for me) is that I added the main “node/123” URL and also the main URL alias to the keywords. Which I take advantage of next.

Fixing up the URLs

Drupal provides URLs as just the identifier into the node table, like “node/123”. I had used some modules (i.e., “pathauto” and “URL Alias”) to provide nicer URLs that looked like “2008/03/30/drupal-to-movable-type”. Movable Type provides URLs as “2008/03/drupaltomovable_type”. This works well to allow me to provide some redirects to the new URLs. To do this, I’ve imported the URLs as keywords (as mentioned above) and then added the following lines to my .htaccess files in my new blog sites:

RewriteEngine on
RewriteRule ^(node/\d+)       /mt/mt-search.cgi?search=$1 [R,L]
RewriteRule ^(\d+/\d+/\d+/.*) /mt/mt-search.cgi?search=$1 [R,L]

Now, whenever someone hits one of the old links, they are redirected to a search, which will find the post they were supposed to hit. This worked out pretty well.

Moving the RSS Feeds

In the process of moving to the new server, my RSS feeds changed names. I had five different feeds. One for each site and then one that summarized all of them. I’ve decided to start using FeedBurner to host my feeds. So, after I switched to Movable Type and setup my feeds, I then setup more rules in the .htaccess file to redirect anyone who might be reading my feeds.

RewriteRule ^rss.xml            http://feeds.feedburner.com/TildeSterling [R=301,L]
RewriteRule ^frontpage/feed     http://feeds.feedburner.com/AndrewSterlingHanenkamp [R=301,L]
RewriteRule ^contentment/feed   http://feeds.feedburner.com/Contentment [R=301,L]
RewriteRule ^openscripture/feed http://feeds.feedburner.com/AndrewSterlingHanenkamp [R=301,L]
RewriteRule ^gabe/feed          http://feeds.feedburner.com/GabrielScottHanenkamp [R=301,L]

Now, all my feeds are permanently redirected to their new locations at FeedBurner.

The Design

The rest was the design work. I’m not going into detail here. Just look at it. I did it. That’s me and my shoddy Gimp/Photoshop work.

That’s It

That’s pretty much it. I’ve got some other stuff I’m planning, but that will probably take months more to get there.

I was scrolling through my morning feeds and came across this gem on Slashdot, "Should Schools Block Sites Like Wikipedia?
" The story goes on to describe someone's question regarding the fact that the local school board decided to block access to Wikipedia "because students may be exposed to misinformation". This simply describes an opportunity lost and a new form of book banning.

I grew up learning about how books like the Wizard of Oz were banned from schools because some people didn't like the fact that it had a witch. Other books were banned because they praised opposing political ideals. Now, apparently, it's vogue to ban something because it might not be accurate. However, this is still just as bad an idea as all the original book bans.

Where do you stop? The New York Times has in the past few years had at least one reporter fired because he fabricated information published in the Times. Should we block the New York Times because it might expose students to misinformation? Should we also block cable television because certain news anchors published stories in that medium without verifying the facts? Should we start blocking the text books because they sometimes contain errors or misrepresent facts?

Should we stop allowing teachers in the class room because sometimes teachers share incorrect informatino? I once had a teacher tell me that NASA was experimenting with mounting monkeys heads on robots and that the monkeys were controlling the robots successfully, but only for a few hours until the head died. Should we throw all the teachers out because there are some that are crackpots?

No! Of course not. This is an opportunity to explain that all sources are suspect until they are corroborated. If you read something in Wikipedia, you take a note to check any fact you can't verify from your own experience. You then verify that fact in the quoted source (for articles that properly quote sources), you can check your local library and do a database search, check another encyclopedia, look in a journal or magazine, find related books, etc. I find that Wikipedia is a great place to get started, but I certainly don't think of it as authoritative.

On the other hand, you should always do the same thing, insofar as you are able, for any source of information. If a source of information makes a claim, you should check it's sources and possibly verify the claim in other places. It's easy to make an unverified claim. It's hard to make a claim that is cited and backed up with facts and also backed up by other unrelated sources.

The lesson is that some sources are more trustworthy than others, but none are beyond suspicion. The lesson should be that you should always check your sources and verify that what you're reading is factual and reliable. Banning a source only means that you take this object lesson away and you raise students that are actually less able to think for themselves and more ignorant. Great, that's just what we need. Good job Anonymous School Board.

Cheers.

Cross-Site Drupal

I blog on a wide range of topics. Because of this, my blog one day might focus on Bible study and the next on a trip I took and the next on some site improvement I'm making on one of the web sites I maintain or help maintain. I'm guessing that some visiting my site probably get whiplash and probably have a hard time knowing what to pay attention to.

Solution: Crossite module. I've written a module I've dubbed "Crossite" which uses the multi-site features of Drupal to share nodes between them.

This solution provides a kind of ultra-cheap aggregation of information on the various sub-sites. All the sites share the users, nodes, taxonomy information, but have their own themes, settings, and cache. The main feature provided by this plugin does two things. First, a node is categorized as "belonging" to a given site based upon the taxonomy terms it has been filed under. If a node has no matching taxonomy terms, it is treated as if it belongs to the default site.

The second half of this functionality is a redirect that is performed when viewing nodes that belong to a different site. If you click on a link that leads to a node page for a site that belongs to another and does not belong to this site, you will be redirect to the node's page on that other site. A node might belong to multiple sites or all sites simultaneously if it has the correct terms attached to it. In fact, one term might even link a node to multiple sites.

A secondary feature is that it adds a new property to node objects, named "sites". This returns an array of sites to which the node belongs. This can then be used to determine how to theme pages or do other custom tid-bits.

I'd like to make a "Crossite_view" module to distribute with the parent module to further facilitate this by allowing sites to have specialized list views. The configuration of the system is a kludge and needs to be incorporated into the UI to make installation easier/possible. Right now, you have to modify the "settings.php" file for all the sites. Anyway, I'm waiting for my CVS account on Drupal.org before publishing the module.

In application, I've expanded my web site now to include two domains I've owned for awhile but had been hoping to use for other purposes. I don't think those other things are going to happen just now, but I wanted to use them in the meantime.

This site, Andrew.Sterling.Hanenkamp.com will focus on the "life" stories about things happening with me and my family, my general opinions, etc. I will also use this site as the primary aggregation site for all the other articles. The main list will feature those articles as other headlines between the "Life" articles. I will probably provide a master feed of all my web contributions as well. I'd like to have something like the Newsvine seed where I can pull in articles, comments, and other bits I post on other sites as well---such as links to my Boomer Bulletin articles.

I will feature technical stories about computing, Perl, Java, programming, web, software, Apple, Linux, etc. on Contentment. That site formerly hosted my CMS efforts, which are now defunct. I had, briefly, also planned on using it as a place to discuss church site development or provide project management tools to teams building church web sites. However, one is already being handled by the Churches group on Drupal Groups. The other proved to be too ambitious for me to do in the time I could spare between everything else I do when not at work.

My Bible studies will be featured on Open Scripture. This site reflects my desire to know my Bible and "Open Scripture" describes a kind of mini-motto that describes my take on the Christian life. I purchased this domain in the hopes of helping create some kind of Bible study/collaboration tools. Again this has proven to be too ambitious and probably premature. In the meantime, I will use it to share my personal Bible study.

My web site continues to evolve, bit by bit. It's kind of a fun little project to dink with every few weeks and is enough to keep me entertained at home.

Cheers.

Enabling the dream

Anyone that knows me should know that I'm often goofy, but not often really happy. I really feel frustrated and grumpy, a lot. If folks would just listen and do things the right way, I wouldn't feel this way. Seriously, that's how I feel a lot of the time. Yet, I occasionally get sappy. Prepare to be sapped.

I got a job working for Boomer Consulting, Inc. in February. I develop the software that runs the web site, or at least integrate and hack it. Prior to getting this job I was feeling really stifled at K-State. I liked what I did. I liked the folks I worked with. The pay wasn't terrible. The perks were alright. The medical plan was excellent. Yet, I felt stifled. I'd started losing interest. I was getting more frustrated than usual. And I wasn't doing the thing I like most: building web sites.

Moving to Boomer has allowed me to engage in my favorite occupation, but it also has done something more. It's freed me to think about things I hadn't really dealt with for months. I'd started to let the New Hope web site slide. My own web site practically vanished for months. My ~Sterling site went down and still hasn't come back (I still plan on it, by the way, for holding my resumé and such), and I'd really gotten kind of burnt out on everything.

I'm not sure what it is about the move to Boomer, but a great weight has been lifted in the process. Of course, the weight of deadlines and hard work weigh down on me more than my last position. I think the other part of this that has shifted the tide is that I've given up on the frivolous plans to build a CMS. I have new plans for contentment.org, but those plans don't feel nearly so urgent as my plans to build Contentment the CMS.

Anyway, I'm feeling really good about the things God has done in my life over the last six months. I've started getting back into my Bible again, praying, and taking time to do things besides stare at a screen 20 hours a day (I think I'm now down to 17). :-P

Cheers.

Okay, so I'm announcing here the first mini-launch of the New Hope Church web site. The site is now running on a later version of Drupal and should have most of the old features intact, with a very few additions. I need to get the login information to Eric so he can start on the theming now so we can have the full launch.

I'm most excited right now about the new features that are coming rather than the few I've added. I'm going to talk about both and the features we already had as a reminder.

Existing Features

The New Hope Church web site has featured these capabilities to date:

Blogging
Any staff member or ministry leader in the system has had the ability to create a blog within the site. However, a very limited set of members have been given the appropriate permissions and of those, they generally have blogs elsewhere or not at all.
Comments
Any authenticated visitor to the site may add comments to most of the stories, audio messages, and events on the site. Again, however, this has never been used. It has never been advertised and I don't know if anyone has noticed the comment links since the current site design works to hide them.
Contact Forms
Each staff member has a contact form on the site. This was the only available way of contacting staff members during the previous iteration because I never implemented a better way. The main problem is that the contact forms require registration to work.
Events
This was one of the most heavily used features. The only serious issue with this was that recurring events weren't possible, so we couldn't add events like the Fusion youth meetings or Sunday gatherings unless we wanted to add each and every occurrence. I don't think anyone on staff has that much time to spare.
Messages
This is the single most used feature of the site. We upload audio to the site and then visitors can read notes about the sermon, download the audio, and listen to Podcasts.
Map
There is a map on the site showing the location of the old church office, Flint Hills Christian School where we meet for on Sunday, and the location of our land. Unforunately, an IE bug I documented a while back keeps that from working properly for anyone using IE.
Notifications
Staff, ministry leaders, and administrators (i.e., me) can setup email notifications when the site changes. I originally use this to watch for when Tony uploads the audio and to check back to make sure things were working. However, I confess that I've been simply deleting the notifications lately.
Pages
Various informational pages have been placed on the site. So far, this isn't much used, but I think that we'll see the amount of information on the site increase over time.
Biographies
Each staff member has a bio on the site associated with their user page. Any member of the site may also have a biography. More information could be added to the member profiles in the future, but I have no direction on what kind of information folks would like to see.
Search
The Drupal search abilities were a little pathetic in 4.5 and 4.6, but it looks like things are quite a bit better now in 4.7.
Announcements
In addition to events, some announcements don't have a specific date associated with them. These are heavily used in the current system and this should continue.
Categorization
Currently, categorization is only used to differentiate between which announcements and events belong to which ministries. With this update I've replaced the original categorization system with a different system that I think will serve us better.
Throttling
The site does have some throttling capabilities. That is, when it starts to see heavy loads, the site will start disabling some features of the site to keep it from being too overloaded. I've not configured this very well yet, so it probably needs to be looked at some more.
TinyMCE
No one visiting the site needs to know HTML. All pages can be editted using an editor with a Word-like interface.
Upload
Staff and ministry leaders can upload files to the site to associate them with stories, pages, events, etc. Unfortunately, our hosting service limits uploads to under 10MB, so this doesn't always serve us very well.
URL Filter
All documents on the site will automatically convert unlinked URLs and email addresses into links. This is also meant to simplify how the site is used for the non-tech-savvy.

Okay, that's what we already had. Now, on to what has been added.

New Features

Here's the list of new features that I've added with this update of the software running the web site.

Categories
I've switched the system from the "taxonomy" plugin to the "categories" plugin. One of the main reasons Drupal is popular is it's "taxonomy" system which provides a very rich language for describing metadata. Metadata, in this case, is basically just extra keywords associated with any particular page on the site.

For example, an announcement might have a taxonomy term associated with it named "Announcement" in the "Section" vocabulary. It might also have a term of "LIFE Groups" associated with it in the "Ministry" vocabulary. Each term is a particular keyword and each vocabulary is just a set of related keywords. The taxonomy system is very nice, but it has a shortcoming in that you can't describe the terms themselves very well. For example, what are "LIFE Groups"? What is a "Ministry"? These might be legitimate questions we'd like to answer and the taxonomy provides only a very rudimentary solution to this problem. I used some weak solutions in the previous version of Drupal to address this problem, but they didn't really work very well.

With the latest version of Drupal there's a new module called, "Categories" that promotes each vocabulary to a first-class citizen called a container and each term to a first-class citizen called a category. Each of these have the same full power as a regular web page and can contain a full description of what they are. This is a cool enough feature, that I will probably start using it on this web site soon. Kudos to Greenash for pioneering this.


Repeating Events

This resolves the old problem of not being able to have recurring events on the calendar. We can now create an event for Sunday morning worship and then make it a recurring event that repeats to a certain date. Handy.

Feedback

In addition to the Contact forms already in place. I've now added three new feedback forms that do not require creating an account to use. This will surely mean a little more spam for those of us on the receiving end, but that's not easy to avoid. There's now a form that is kind of an online "white card" like we pass out in church, a form for prayer requests, and a form for site problems.

Google Analytics

The chuch site has started collecting more statistics about visitors to help us improve the content of the site. I used Google Analytics because it's free, though, we've had some worries about it's accuracy at work. I'm hoping it will be better than nothing.

Just in case, I have also added a couple other stats modules to Drupal as well, to use Drupal's built-in statistic gathering capabilities to track things directly from the site too.


Google Sitemap

The site will start generating a Google sitemap, which should help improve how our search results show up in the most popular search engine.

Menu

This is a de facto improvement that comes just with upgrading 4.7, but we can now create much nicer menus in the system than we could before.

Automatic Path names

No more "node/123" links! I've added this in to automatically create easier to read and link paths within the system. Anytime someone joins our site, they're user profile will be at "user/user_name" rather than "user/123". Similarly, whenever an event is created, it will be at "calendar/2006/08/13/some_event_name". There are still a few irritations to work out of the names being generated, but I'm much happier with the way this is working already. By the way, the old links still work too, so they won't be broken either.

That's what's already new. Now on to the stuff that I think's going to be freakin' cool.

Upcoming Features

Here's my wishlist:

Google Map Improvements/Location/Geocoding
I've already started playing with this a bit. One thing a lot of folks have asked for is an online directory. I've been thinking about how to do this and I think I can provide an online directory for members that will not compromise anyone's privacy.

In addition to providing a directory, I will be able to provide a map showing the map location of all our members on a single map. So, you'll be able to see how close you live to fellow New Hope members. This will probably include a more public way of publishing LIFE group and event locations as well.


Forums

I would like to add forums to the site. Drupal 4.7 now includes the ability to create a pretty decent forums system directly in site. I think we could have various forums for things, particularly site suggestions and problems. This will be combined with my next item...

Organic Groups

This is a feature added by the CivicSpace crowd to allow members of a Drupal site to create groups on an ad hoc basis. With some controls on the system, we could allow staff members and ministry leaders to create mini-sites in the system with their own stories, events, audio messages, forums, etc. I see this as being a really exciting feature for the web site with the potential to revolutionize the communications abilities of thte staff and other leaders.

Okay, that's not the full list of what I want to do, but that's plenty long enough. It will probably take some weeks before any of the latter bits are implemented and I'm still looking for and waiting to hear bug reports on this latest update.

Cheers.

I'm going to make this quick because Terri needs a counter-weight in bed (we have a waterbed) and I've already wasted much time responding to Slithy on Brent's blog on the history of sexual promiscuity in western society.

I've started the module update and have gotten most of the replacement modules installed that can be installed. However, I've had to get a replacement for the taxonomy_assoc module (as I thought I probably would), since that module has been discontinuied in favor of the Category module.

The Category module is a drop-in replacement for Taxonomy that replaces vocabularies with "containers" and terms with "categories". The difference is that both of these are now nodes. Not only this, but any content type can be treated as a container or category, effectively giving you a node for every term. Unfortunately, it's not quite as flexible in some ways as it doesn't seem to have a "folksonomy". Though, I haven't yet looked at it far enough. If it does have tagging or it can be added, I will probably migrate this site to it as well.

This module also replaces the Book module, but I haven't really looked into that since we don't use the Book module for anything at New Hope yet, though I think we might after talking to Dan and Tony a couple weeks ago.

Other than that, I've also added, but haven't tested the eventrepeat module. I've updated the various other modules that are available that I mentioned previously. I also need to see if Tony wants me to do anything to add image galleries, forums, and some other pieces. Finally, I think things are now to the point that Eric can start working on the design whenever he feels like it.

Cheers.

I've started the software update for the New Hope web site. I've got the main Drupal system updated, but I haven't started on the modules. I'm doing the updates to a copy of the site so the current site can still run in the meantime (so don't bother looking for changes as they won't show up until we're done).

The trickiest bit is going to be that some of the modules we use on the current site are used because Drupal was lacking certain features that it no longer lacks. The other tricky bit will be the modules that are being discontinued in favor of better solutions--i.e., I'll have to do some conversions.

Here's the list of modules that need to be updated:

  • event. I also plan to install eventrepeat as part of this process so that we can start showing recurring events, like Sunday mornings and youth meetings.
  • flexinode. I'm going to replace this module with the content construction kit if I can. If it will be too painful, I may leave this for an update in a month or two.
  • form_mail. I don't know what to do with this. I don't know what is available for this functionality.
  • forms. Ditto.
  • notify. This will either be an upgrade or I'll use a better notification module (not that there are many good choices for this functionality, according to my recent research).
  • sermon_customizations. I hacked in some functionality to make Podcasts and some other bits to work correctly. Most of this can probably be ripped back out.
  • taxonomy_assoc. This module associates nodes with vocabularies and terms. This is used in some important places, so I'll have to see what to do about this one (or just update, if possible).
  • taxonomy_context. I can't honestly remember what this does at the moment, but I think this is a ditto of the last comment.
  • theme_editor. I'll have to find out from Eric if this is still needed. I may just turn it off.
  • tiny_mce. This is important. This makes it possible for our staff to post to the site without knowing any HTML or special document formats. I'll probably need to discuss with other stakeholders what features to include in the future too. There has been some disagreement about which functionality should be available to authors.
  • urlfilter. This is used to turn http://... strings into links when someone doesn't link a URL. This is probably a trivial update.

The last bit that I'll need to update is making sure the script I've written to allow Tony to upload recorded audio also gets a face lift. It'll surely need a few changes. I wrote this script to get around upload restrictions set on PHP code on our host and to help standardize the MP3 tags in the audio for the Podcasts.

That pretty well summarizes my plans for the next launch. Eric is going to be responsible for importing the design that Jay Risner has put together for New Hope. I hope we can get the first release knocked out by the first week of September. I'd like it to be before the students return, but that may not be realistic given the data conversion I'll have to perform and some of the remaining design questions Eric has. We'll get it done.

Cheers.

Drupal: The glory days

My web site was never so popular as it were when it were Drupal. Now I'm back to the glory days. Well, maybe. I lost a lot of prestige in the months that I quite maintaining my blog very well while I was attempting to build up Contentment. It also hurts that the paths I had have changed and broke and are incorrect. However, I now have better paths due to my friend, pathauto.

I've now finished importing all the content from my previous WordPress and Drupal sites. I did the blog posts by hand because many of them needed editting and I took it as an opportunity to remember the olden days. There's still more work to be done if I'm really going to get the archives correct since some links have changed and some of the text encoding is screwed up in a few places still, but it may just stand as is.

I've been thinking of bringing in my Blosxom posts as well and even thought of going back to the stuff I built in Everything and back in the static HTML days of yore, but if that happens it probably won't be this month. ;) I'm tired of imports, but I may end up doing it.

One of the coolest new custom features I built this time around was the Categories block to the side there. The most exciting aspect of that is how I order the categories so that both quantity and how recently the tag was used. I built a corresponding Popular Tags page describing how the process works containing a complete list and shows individual scores. It's not perfect, but it's relatively nice.

I still need to tweak the design more. Comments look terrible and I'm not very fond the sidebars. They should match the curviness of the banner. I also need to tweak margins because there's not enough whitespace. However, I wanted to launch as soon as possible because it's a drag having a sucky blog...at least for me.

Cheers.

Popular Tags

This is the list of all tags I've used in my blog to those point (this includes or will include tags used in content aggregated from other sites as well).

Categories

$container_id = 181;
$sql =
"SELECT d.nid, d.title, c.description, ".
" MAX(n.created) AS updated, ".
" COUNT(*) AS count, ".
" SUM(2.5/LOG(0.25*((UNIX_TIMESTAMP()-n.created)/2592000)+1.5)-1) AS score ".
"FROM {node} d ".
" INNER JOIN {category} c ON c.cid = d.nid ".
" INNER JOIN {category_node} cn ON cn.cid = c.cid ".
" INNER JOIN {node} n ON n.nid = cn.nid ".
"WHERE c.cnid = %d AND n.status = 1 ".
"GROUP BY d.nid, d.title, c.description ".
"ORDER BY score DESC";

$count_sql = "SELECT COUNT(*) FROM {category} c WHERE c.cnid = $container_id";

$result = pager_query($sql, 30, 0, $count_sql, $container_id);

while ($category = db_fetch_object($result)) {
$items[] = array(
l($category->title, 'node/'. $category->nid,
array('title' => $category->description)),
$category->count,
round($category->score, 1),
t('%time ago', array('%time' => format_interval(time() - $category->updated, 3))),
);
}

print theme('table', array('tag', 'count', 'score', 'last update'), $items);
print theme('pager');
?>

How it works

The list is sorted in order of popularity using an inverse-logarithm scoring method I developed myself. Basically, the most recently used terms get a very high score that tapers off to a relatively low score once the post is a year old or older. Each use of a term is cumulative so the more commonly used terms will appear higher on the list even if they haven't been used as recently as others. This list will sort itself as time goes on according to whatever my latest posts are about.

Originally, when I used the built-in taxonomy module of Drupal, I used this SQL query to do the calculation:

SELECT d.tid, d.name, d.description, 
    MAX(n.created) AS updated, 
    COUNT(*) AS count, 
    SUM(2.5/LOG(0.25*((UNIX_TIMESTAMP() 
        - n.created)/2592000)+1.5)-1) AS score 
FROM {term_data} d 
    INNER JOIN term_node USING (tid) 
    INNER JOIN node n USING (nid) 
WHERE d.vid = ? 
    AND n.status = 1 
GROUP BY d.tid, d.name, d.description 
ORDER BY score DESC

However, since I recently switched to the Category module, I updated the SQL to reflect this with:

SELECT d.nid, d.title, c.description,
       MAX(n.created) AS updated,
       COUNT(*) AS count,
       SUM(2.5/LOG(0.25*((UNIX_TIMESTAMP()-n.created)/2592000)+1.5)-1) AS score
FROM {node} d
     INNER JOIN {category} c ON c.cid = d.nid
     INNER JOIN {category_node} cn ON cn.cid = c.cid
     INNER JOIN {node} n ON n.nid = cn.nid
WHERE c.cnid = ? AND n.status = 1
GROUP BY d.nid, d.title, c.description
ORDER BY score DESC

Here's the function describing the math involved. I developed this function for calculating the score by arbitrarily modifying the log curve to suit my needs.

s equals the sum iterating over eye from one to en of two-point-five over one-point-five plus em log zero-point-two-five quantity minus one

In the equation, s represents the final score, n is the number of terms and i is the iterator over the terms. The mi represents the number of months since the creation of the post the ith use of the term belongs to. Since Drupal stores time in seconds since the epoch (i.e., January 1, 1970), the current time is calculated by subtracting the node's creation time from the current time and then dividing by 2,592,000, which is the number of seconds in a month.

If you view a curve plot for an individual iteration, you would note that when the delta (mi) is 0, the score for that term will be around 5.2. When the delta is 12, the score is about 1.

The Pounce Effect

This is something I've noticed for a very long time and I'm a bit glad that my friends aren't of the tendency to engage too greatly in this stupid side-effect of the blogosphere. Whenever there's a news item that has any single sentence in it that can be used to make hay, the bloggers pounce on it and beat that little sentence to tiny bits. This is magnified when a whole article comes out that places certain issues or individuals in a bad light, particularly everyone's current least favorite President.

I'm not particularly pleased with President Bush myself, but to nit-pick everything he says and does because you don't like him is a little pathetic. A lot of conservatives did the same thing to President Clinton when he was in office (I may have even done it a bit myself), but that's no excuse.

This came up because I was looking through my Newsvine feeds and noticed an article pouncing on another article. The pouncer makes an inane comment trying to say that the economy must suck since the markets bounced because of a change in the Treasury Secretary—a "trivial event." I won't try to debunk that one because it's not my point, but I do think she's dead-wrong.

All one has to do is search for "George Bush" and the majority of the hits are either favorite pouncees or top pouncers. It'd be much nicer if such a search yielded substantive and informational hits rather than Bush-bashing. Bush/Clinton-bashing may be a fun hobby, but it doesn't really make for very interesting reading.

This is also not limited to just the President. News articles on anything scientific or mentions the term "intelligent design" usually gets an evolution pounce. The same goes for the separation of church-and-state, abortion, the war in Iraq, et. al.

While I'm not offering any real solutions to the issue, I will make a plug for sane discussion. The key is not to be passionless, but to back up everything with fact and to avoid any specious or emotional argument. If you want to say that the economy sucks and is evidenced by the dip in the markets during a change in the office of the Secretary of the Treasury, by all means say it, but back up the statement with fact and sources. Otherwise, you just add to the noise.

Cheers.

Some of my friends may know of how I have over the last year or so become somewhat attached to producing content for Wikipedia. However, my friend and coworker, Eric, (unfortunately his web site is EMPTY, so I can't really link to him) recently introduced me to Newsvine.

It basically just provides AP articles with the additionally ability to vote on these, add comments, create columns, and provide "seed" links to other news found on the net. I've popped over there a couple times and I've been very pleased with the experience so far. The community isn't superhuge yet and the site is still relatively immature (only a year old), but I see a lot of potential in how I've already been able to skip around the site to read articles, find related information, and discuss.

Anyway, I could see this being another place I hit whenever I need a break from the mundane at work. I could also see some of my political opinions being expressed there rather than on this blog. I have also added this to my list of sites to watch for new innovations in community development and we might consider using a few of the more interesting ideas in some of the community aspects of the new Boomer web site.

Cheers.

Okay, so I had a suspicion when I took this job this would happen. I even prophesied saying, back in the day, "If I ever got a job doing what I love to do during the day, I'll probably stop dinking around so much at night." Well, it has happened.

Interaction Developer

I now have a job for Boomer Consulting working as Interaction Developer. While my language of choice is Perl and I have a few modules on CPAN and this job is Java, I am still working to help folks use the web in ways that make their lives easier. This is essentially what I've wanted to do for about 5 or 6 years.

I thought I would miss dinking around with the IT toys that I spent my life on in my last job, but I do not. In fact, when IT has come up here, I have done what was required but haven't really felt excitement about it. I am an analyst, designer, and programmer now. I love this job.

We're Pregnant!

That's such a stupid line. "We're pregnant." I knocked up my wife and she's pregnant. I'm just watching off to the side while her hormones bounce her around like a superball thrown too hard in an enclosed space. Anyway, this makes me quite ecstatic as well. I've been looking forward to having kids since we got married and the adventure has just begun.

What this does mean is that I'm going to have a lot less "free" time at home. That is, my free time will need to be spent a little more focused on upkeep on the yard and house, watching the rugrat, spending time with my wife, and focusing attention on the being a pastor for my family. One of my previous bosses, Terry, once explained how he used to have hobby project once, then he had kids. I don't expect I'll lose the ability to sit and read a novel or dink around with something on the ol' compy at home, but it'll certainly scale back the time available for such persuits.

I'm Tired

I'm a bit tired of dinking around with stuff I never finish. I have learned a lot about myself and my predilection for toying with a problem until it's dead. I'm never satisfied with a solution and always need to dink with it more. I really think I am close to where Contentment would be a nice tool, but I don't think I'll really be able to push a project like it by myself, even if I got it done in the next six months.

The Solution

Thus, I'm revamping or starting up each of the sites I've been planning for the last couple years.

I'm recreating my personal blog site using WordPress so I'll be able to blog, which I've sorely missed for the last few months since I redid this site last.

I'm going to also reuse Contentment.org for a new purpose, which is to create a site specialized towards discussing content management and web development issues. I'm not sure how exactly to go about it yet and am currently considering a Wiki, but more thought will be made before the site is actually built. It will probably either run via MediaWiki or the Joomla CMS. More to come on that.

I'm also starting up the site I've been most looking forward to for sometime: OpenScripture.net. This is going to become a site dedicated to learning about Christ and trying to answer the deeper questions of life and Christianity from a layman's perspective. I'm going to be talking to various friends and others to get it started. I want it to be based around a community of men (at first, but women too), who passionately want to know the God of the Bible and are willing to share there thoughts and discuss with each other in an open forum.

Anyway, that's where this all leads. I welcome comments on this and, if you're a friend, I will probably be asking for your input shortly on one, the other, or both of my new additional sites to see if I can get this done right.

Cheers.

Neat. ~Sterling has been rebuilt using the latest CMS to take the Open Source world by storm...er something. Okay, so the "secret" project I've been working on for years is finally doing something because I decided to stop being "elegante" and decided to JustMakeItWork(tm).

What have I done? Well, I wrote a little tool to help me manage the CIS Support Site for work. This little tool is a combination themer/indexer for static pages. It also does some on-the-fly generation of HTML from reStructuredText, which is what we write most of our docs in. It seemed pretty useful and is similar to the software I was using to run this site before October 2004, called Blosxom, which is a lightweight file-based blogger.

Anyway, when I had some trouble getting access to K-State Online for my course last semester, I decided to try and dump my course data into it on ~Sterling. With a few modifications it worked quite well and quickly supplanted any previous ideas I had about the content management systems I'd been toying with for the past several years.

Most of the work is already done by HTML::Mason. My system just took advantage of the features already present to add indexing and theming and generation of content from other formats. ~Sterling took it a bit further by adding the ability to generate even more complicated content (especially, ripping apart zipped Keynote "files" and using XSLT to generate HTML outlines).

At this point, I ran into a few issues:

  1. Adding new generators was requiring lots of custom code and my indexing code was becoming convoluted.
  2. New content had to be added with care, otherwise Mason would try to interpret files it had no business touching. When this happened, my indexer would basically bring the entire site to it's knees with a single exception.
  3. Some content is just better stored in a database. Blog entries, news items, and simple records are just a few examples. The system had no way of coping with any of these.

Thus, since about the time I put Drupal on this site, I've been working on a replacement. Drupal is merely a temporary expedient. I started completely from scratch, but have dragged in a lot of the bits from the existing "knowledge base" system to build this new system, which I am dubbing as Contentment (superceding all the predecessors I'd created and called this).

This system currently features a lot of unused features, but most of the good ones are employed currently. One of the best features I just added this week and after just a few days it's practically remade the quality of the system. Specifically:

  1. It features a (largely unused, as yet) forms handler that can help design forms and wizards with a fairly small amount of effort. I borrowed a lot from the kinds of work that Everything has done in this area.
  2. It uses the SPOPS object-mapping system to provide a database API. It's not required that new plugins use this API, but all the existing database pieces use it.
  3. The system automatically provides for context, sessions, and logins. The user accounting system is completely pluggable, so new support for LDAP or other login types could be added with a little effort.
  4. The system provides a basic permissions system. All of these features have been designed to make adding database-based plugins possible, but there really aren't any yet.
  5. The major feature that has really made the system work despite the lack of any database plugins is the VFS system I've put together. I've debated whether or not this should be forked into a separate project, but I'm going to leave it where it is for now. Anyway, this enhances Mason's abilities by quite a bit and allows for a much more general way of looking at files. This way, Mason no longer has primary control of generating files, but passes that control off to other plugins.
  6. Right now the system works via CGI, but I'd like to put together a mod_perl front-end to take advantage of those features. I've designed everything to this point with mod_perl in mind, so it should work with minimal effort.

That's a pretty bad mish-mash summary of the features. There's a lot more I could say, but I'll save that for documentation. I'm going to admit that I've had a SourceForge project for this for eons, but that it'd never really worked until now. I'm so excited about how this is going now, that I have registered Contentment.org and will be posting information and documentation there. I'm going to, for now, use the mailing lists, bug trackers, announcements, and CVS repository at SourceForge. (Though, I'm hunting hard for a way to keep it in Subversion as I strongly prefer it, despite it's performance and other issues.)

Anyway, I wanted to announce that and say that Drupal may be saying farewell to this site soon---if I can get the plugins written and translate all of my Blosxom and Drupal entries into my new plugins.

~Sterling is Coming Along

For the last few weeks, I've been working on my work web site, ~Sterling to improve it's capabilities. The purpose of this update is many-fold. Read on if you want to learn about my software (you may want to read this post first)...

My web site is based upon the "Knowledge Base" software I developed for the CIS Support Site. This has made keeping this web site up-to-date considerably easier. The software started as a handy index generator that generates indexes of pages posted into the system automatically. Additionally, it's able to perform some basic transformations on input files to generate HTML or RSS or whatever. It's a very minimal content manager that mostly relies upon the features of Mason for most of its functionality.

Okay, so I branched the system to create ~Sterling, which was the repository for the CIS 450 web site last year. I added some more features to make it automatically detect multiple file formats with the same basename minus suffix (e.g., Session-01.pdf, Session-01.pdf, Session-01.key.zip). This made it possible to view multiple versions of a file and using the transformation bits, I was able to generate HTML summaries straight from my Keynote presentations since the format is based around an XML file.

Problems: (1) I now have two versions of the "knowledge base" software that I wanted to have the same features. (2) Adding content always requires dumping files into the system, which is overkill for items like blog/news entries, blog aggregation, etc. (3) One bad file spoiled the whole web site—any index page that found a badly formatted Mason file caused the index to puke. (4) The transformation system was a kludge and required careful tweaking and depended dubiously on file suffixes.

Thus, I endeavored to rewrite the software and have now decided that it is satisfactory for my content management dreams of the past several years. It's not quite ready for distribution as I need to add a little bit more documentation, but I anticipate that in the next month or so, I'll be updating the woeful Contentment project page.

I should be able to take the existing CIS Knowledge Base and drop it into the new system with very little effort. My ~Sterling page is going to be repopulated over the next couple weeks with the original files, but using the new improved transformation system. I still need to migrate the indexing system from the old knowledge base to the new one, but that should be a relatively simple matter (and this time, errors will be handled gracefully!)

The new transformation system is really the key. I borrowed a lot of ideas from Cocoon since I've always kind of liked that system. Basically, each file in the system is first checked to see what input "kind" it has (determined by a set of Mason plugin components, which pick the "kind" from file suffix, file contents, etc.). Based on this information, the file is run through a "generator," which translates the file into an initial kind. The system only has two generators right now: Mason (runs the file as a Mason component) and the fallback generator which just reads the file as is.

Then, the transformation system is applied to each input file. The transformation system attempts to find a sequence of transformations that can be applied to the file to get it from it's initial kind to the requested final kind (which is determined by more plugin Mason components, usually based upon the URL or query parameters). If no transformation can be found, then you'll get a 404, otherwise, it attempts to find the best transformation using a shortest path search (which is probably too costly, but works fine for now with a very small number of transformers). The transformers are applied to get the final output file.

Yet, we're not done yet. Finally, based upon the final output kind, another group of components are applied to the output, called "filters," which further modify the file. The main reason for this is that HTML files coming through the system need to have some links fixed, etc. before output. I had intially thought that this would be a good place to put the theme engine too, but I've decided against for now until I can come up with a decent policy for regulating how themes should work. I have a theming engine in place now, but it depends upon the "autohandler" feature of Mason, in the same way as the previous version of the knowledge base.

Once all the filters have been applied, the file is finally output (after being passed through any autohandlers, as per normal Mason operation).

I should have the system adapted to take any index and turn it into a proper RSS feed and I also want to add Atom feeds this time round as well (and better put in the "alternate" links to let browsers know it's there).

The next step is to add in the database features so that the web site can store some content into MySQL. This will make a lot of the routine updates to the site much easier. A file manager would also be handy to allow users to upload and manage files through a web browser, which is another goal of mine.

If all goes well, I may be dropping Drupal for Contentment in a couple months. I won't hold my breath though...

~Sterling, TNG

I'm bored with my CIS web site and a little tired of maintaining it. For anyone looking for it's content, it still exists and I'll be bringing it back, but I'm trying to put together the next generation content management system for my web site.

History: Okay, this is pathetic. I am so picky about my crud that I have literally spent the last 7 years trying to put together a content management framework for web design as my number one pet project. All of that work and I have very little to show for it. In that time, I've restarted at least once a year.

The original reason for all this work is that I wanted to have a journal of articles and essays that I updated on a regular basis, but wouldn't have to maintain the index. I wish I could say that 7 years ago, I started blogging before "blogging" was even a part of Internet jingo, but I did want to do it 7 years ago.

More History: I've been "programming" since I was able to read. Something around five years old. I wasn't really programming per se, but I did know how to copy programs from Compute Magazine. When I was eleven, my friend Lucas gave me a (pirated) copy of Turbo Pascal 6.0. He also had a tutorial he'd gotten off of a local BBS to learn Pascal. From there, I got my one wish for my sixteenth birthday, Borland C++ 3.1 with Application Framework and then taught myself C and C++. (Yes, I have been geek since I was five years old.)

At K-State, because of the leadership of Dr. Schmidt, I learned Java and became completely emersed in the language. It was around this time, while working as a consultant for Network Resource Group, I began trying to put together a CMS. It wasn't called a CMS back then, but that's what it was.

Over time, I got frustrated with the restrictiveness of the Java language. I always felt like I was trying to wrap my ideas around the Java language instead of just executing them. Java is so very verbose, but it does provide a standard library for doing everything. Anyway, my first attempt went up in flames.

I then had my first, brief, flirtation with Cocoon while I was taking CIS 726 as an undergraduate and built a very small implementation of my personal web site with it. Cocoon was cool, but freaking slow!

I then began my trek for a language that could express what I wanted directly. I wanted a language I can think in. I tried out PHP. PHP is real slick. Last I checked, NRG is still using the second web site I designed for them using PHP. We need some login functionality for clients so they could check on the antivirus email service we were planning to supply (though, I understand they've since dropped that service). Anyway, I replaced a web site that took me two weeks to put together (in Java and JSP) with a new one in PHP in less than four hours. Awesome!

Therefore, I embarked on attempt number three in PHP. This is when I came with the concept of a content management system, which everyone else obviously copied from me. ;) This is when I invented, Contentment, as a framework. Unfortunately, my first concept for Contentment was doomed from the beginning. PHP just wasn't elegant enough for me to do anything but RAD. I like PHP and would use it again, but I will not build frameworks in it. Too much spaghetti.

This and the fact that PHP isn't really good for anything but web templates (though, I understand PHP 5 is changing that) brought me back to my search for a language. At NRG, I had messed with Perl a bit for the antivirus stuff (we were extending Anomy Mailtools) and decided to give it a try. I also gave Python a shot. Both are nice languages. Python offers some very elegant scripting features and freedom from static typing. (I can just hear Dr. Stoughton and Dr. Banerjee cringing at that thought.) Python is just too....ugly for me. Yes, I said ugly. It feels clunky and I have to think to hard to do things their way. It's the same old issue I had with Java.

Perl, on the other hand, is like God's gift to the disorganized thinker. I can think in Perl. I write code like I write a term paper. It's easy for me to read and easy for me to write. It fits me. Unfortunately, it too, is very clunky. Many of the language features are hacked on. Imagine an object oriented language where the objects are just things that are "blessed" with a name. Thus, you could have an array or a regular value or a map "blessed" as the same type but have completely different ways of being used. Ick. So, I cope with the clunk because I know Perl 6 is coming.

Anyway, Perl brings me back to my point. I started an implementation in Perl. I failed. I started another implementation in Perl. I again couldn't stand my own code, and started again. Around this time I discovered Mason, which is a really wonderful tool for embedding Perl into source code and is a mini-content management system of it's own, sort of. I tried again with a Perl/Mason combo and again failed.

I then decided, let's go practical. I'll implement something that works RIGHT FREAKING NOW. Ta-da! It worked!....almost. If you go to the CIS Support Site, you can see my handiwork. It works. The original ~Sterling web site featured a forked version of the same software that was improved in a lot of ways from the original. Unfortunately, both of these were still lacking and slowly became less and less manageable—i.e., they didn't scale as well as I'd hoped.

Thus, I have endeavored to use the best of those tools and added new features for database support and a new forms system to try and make the system more scalable. Unfortunately, I've again tried to go too abstract again and not concrete enough.

This time, I'm going to do it right. I'm going to take the best of my third generation Mason/Perl software and my attempted fourth generation Mason/Perl software to build the fifth generation.

This is why ~Sterling is now a blank slate. I'm going to build it up from scratch and I will make it beautiful, or I will die trying. May God have mercy on my soul.

Flirting with Drupal

Okay, I've pretty much lost my mind at this point. This paper is simply killing me. In order to keep myself from going completely insane, I've been taking 15 minute breaks between writing my paper and have somehow managed to get Drupal working.

I've grown tired of Blosxom, so I was looking for a change. Travis has long been a fan of Drupal, but I was previously hesitant because I'm not a big fan of PHP applications in general. I generally consider PHP to be a good prototyping language, but I don't really care for how applications end up looking in the long run—that's a whole other blog, so I'll stop there.

Anyway, I took a look into Drupal 4.5 and I like what I see. It looks like they've done a lot of the stuff similar to the features I like in Bricolage and Everything. So, I'm giving it a shot.

At this point, I like it so much, I'm thinking of recommending it for use on our church site. I still need to figure out how that would look, but I think it's doable.

1

Tags

Find recent content on the main index or look in the archives to find all content.