Tag Archives: Small Axe

Changes to firsttube.com

I am very excited about how portable Small Axe, the engine that powers firsttube.com, has become. I am going to be upgrading the site in the next few weeks. You won’t see a ton of new stuff, but it will be much more powerful and configurable for me.

One place I have made some changes is in the RSS and Atom feeds. Although I advertise my feeds at feeburner, our source feeds are at firsttube.com/feed and firsttube.com/feed/atom. They have received some stylesheet love and are much more readable by the human eye.

Tagged , ,

Export Blogsome, Export Slashdot Journal

I was recently issued a challenge: backup a blogsome blog and the content of a Slashdot journal and merge them into a single database. I foolishly accepted this challenge, knowing that Blogsome is based on WordPress. Come to find out that Blogsome doesn’t allow you to backup or export their WordPress content. Also, Slashdot doesn’t provide you a way to export or backup your journal. The prize was sweet: a brand new, fairly expensive, unlocked mobile phone.

If you want to make a mirror of your blogsome blog, you can use a single very powerful command to generate a snapshot of it from any Linux machine or Windows with Cygwin installed:

wget -k -m -r http://url

But this will only create a static HTML mirror of your website. It won’t allow you manipulate content or put it into another database. That leaves only one way to do it – request the entire site page by page, and parse each page individually. RSS is not reliable here, as most people have it set to only 15 or so items and parsing an enormous page make make PHP or your server run out of memory or alloted script execution time.

It’s a multi-step process, to be certain. It was actually painful to go through the process. Requesting over and over, debugging the script line by line. It takes several steps to get things right. But, eventually, I did it. I was able to export a Blogsome blog in its entirety – every entry, all the categories, all the comments with emails and websites …everything.

Slashdot was the same. It took some tinkering, but eventually, I was able to backup a very lengthy Slashdot journal. Again, in its entirety. I got every post, the date, etc. It was not simple, but it worked well. And I not only got them backed up, I merged them into a single database serving up… Small Axe 0.6 (which was a whole adventure of its own, taking current firsttube.com code and “neutralizing” it). Suffice it to say, when I saw all entries working and served in Small Axe, I had a huge smile. Turns out that the person I was doing this for decided against Small Axe (only because it doesn’t yet offer all the bells and whistles WordPress does, even if it is a beast). But it was irrelevant – the hard part was getting the data properly, and that’s done. Migrating to *any* blog database is possible if you have the time, inclination, and skill to write a SQL export/import script.

Here’s how it works:

* Cycle through each page of the blogsome blog. On each page, we get the entries, the URL, the postid, and other relevant info. We set a flag on each item to 0.
* As we retreive the items, we correct the path to images, spacing, smilies, etc.
* Then we cycle through each page individually. We have all the URLs already, so we go through each one and parse comments. It’s important to know that comment owner comments are marked up differently. As we get the comments, we upodate the flag and let the script run on its own. Our 900+ items took about an hour by meta-refreshing the fully rendered page every 3 seconds.
* As we go through the comments, we strip tags we don’t want, we fix emoticons, we fix internal links, spacing, etc. We must expose emails temporarily if we want them to transfer over.
* Finally, we import them all into a central database with an agreed schema.

If you have a Blogsome blog and/or a Slashdot journal you need backed up, I can help you do it. It’s not a simple process, but it is very accurate and preserves whatever data is exposed via HTML. So for the right barter, I would be very motivated to help. If I can simplify the process, I may create an open script to do this. But for now, I’ve got the code.

On a related note, I’ll probably release an updated version of Small Axe sometime in the not too distant future, because the amount of changing I’ve done and all the features I’ve just implemented are killer. Small Axe is FAR from WordPress caliber tested, but it’s SUPER simple and can do all the basics of a normal blog, including templating, smart per-domain caching, blocking by ip, username, email, or keyword, gravatar support, tons of configuration options, RSS and Atom support, threaded commenting, post locking, post expiring, browser identification, slug-based permalinking, and much more.

Tagged , , , , , ,

Small Axe Getting Close To Actual Releases

I am very close to releasing a flood of code on smallaxesolutions.com. Included in the first batch are:

– Small Axe Encryption (which is VERY breakable (being as though the decrypt code is actually available, but a new version, which uses a password to decode, will eventually be out).

– Small Axe PHP XML-RPC API. YIKES. What a mouthful. It’s by far the easiest API out there. I have searched high and low for an API. Version 0.1 is so simple it’s crazy.

– Dango 2.1. A simple discussion board. Version 3 will be a complete rewrite, so this is a nice mini-step.

Tagged , , , ,

New Admin Panel and Voting

You may notice that there are now buttons next to each entry on my blog. These are “agree” and “disagree” buttons. Although right now each post is assigned a “score,” soon it will probably be “X positives, Y negatives.” You can vote once per entry.

Also, I’m going to be rewriting the administration panel of Small Axe shortly. It’s not because it’s dirty, because it’s not, it’s actually modular and easy to write new plugins. However, It’s not ideally organized. I’ve updated the icons from a mix of random icons and Tango icons to mostly Echo icons from the forthcoming Fedora Core 7. I really like the new ones.

new admin panel

As you can see, I’ve finally added preference panes(!) for some features that I’ve had for awhile, such as User Agent review. The database cleanup link doesn’t work yet, nor does the Recent Votes link, but both are functions I will be implementing shortly.

Tagged , ,

Slug Transition Complete

The transition to a smart, modern, fancy URL system is complete. This is how it works: every item has a title called a “slug.” The slug of *this piece* is “Slug-Transition-Complete.”

Now, your basic operations are read, print, and comment.

So, the URLs work like so:


This requires no tweaking of Apache (other than permitting .htaccess files). There is no mod_rewrite going on here. It’s all in PHP. Huzzah.

Tagged , , ,

Small Axe 0.4 Update

I have frozen the code and made my first updates to Small Axe in a very long time. In fact, calling what’s currently available for download in the -devel build 0.4 is not even really accurate. This build has all the features planned for the 0.7-0.8 timeframe. But alas, it’s “0.4 alpha 1” for now.

It doesn’t work.

Above all things, it doesn’t work right now. The build system is now set to rebuild the demo site every 30 minutes, and the code is changing very quickly, but there are still major problems. All in all, I think I should have it available within the next month or so. All I’m trying to do is fix the features that already work here on firsttube.com. However, they must all be made completely portable, which is a challenge.

Anyway, it’s exciting, because I’ve been working on a lot of stuff I hope to release by year’s end. It’s good to be coding again. Feels good.

God, I’m a nerd.

Tagged , , , ,


Okay, I think I’m pretty happy with the code running firsttube.com. It’s not perfect, there are some things I’d like to do to it, however, I’m going to declare a “freeze” and package this as Small Axe 0.4.

Now, in truth, this build will have all of the features I had planned for 0.4, 0.5, 0.6 and probably 0.7, but since I haven’t released anything since 0.3, I think I’ll tag it 0.4. Then I’ll try to get to 0.5, a bug fix build, quickly.

0.4 will be another MySQL only build. I will reinstate the flatfile and SQLite support as we approach 1.0.

So let’s aim for a late September 0.4 final build. There’s A LOT of work that will have to go into making this code portable again, so I’m going to take my time.

Tagged , , ,

Thread locking

Although its been present for a long time in the database (and to some extent, in the PHP), I’ve never really gotten around to making “thread locking” work right. Until now.

I wrote the piece on Israel earlier because it was on my mind, not because I wanted to start a debate or turn my blog into an anti-Israel platform. So when I saw the tide turning that way, I locked the thread administratively.

It wasn’t pretty, it just spit out undescribed errors. So I’ve finally gotten around to spiffing up the UI, and it now displays on the front page, as well as the comment page, that the thread is locked. It won’t accept new comments.

I’m also doing a quick audit for additional features needed.

Tagged , ,

New Style

As you can see, I’ve made some minor tweaks to the stylesheet for firsttube.com. I am preparing to add the last few features and then stabilize Small Axe for a Fall release.

Tagged , ,


The backend rewrite is nearly complete. The only section left is the Users section, however, since the blog is not really set up security wise for user rights yet, I’m not going to be finishing it for 0.4. I will apply the new backend styles, however.

Anyway, here are some screenshots of the new backend. Click each one to see a larger version:

smallaxe-admin-0.4.jpg smallaxe-new-0.4.jpg

smallaxe-topics-0.4.jpg smallaxe-uploads-0.4.jpg

Tagged , , ,