Planet NEO is killin’ me. Particularly on this low-memory vps.

Planet NEO has close to 500 feeds now. I have the update running hourly still. As I said in this post I dumped all the archived articles from before April 1, 2008, but I’m still sitting at 16,000 posts. So I have two problems.

Feed Updates

The first is when the feed updates. Due to the number of feeds being pulled, it’s taking a long time. And when the cron job runs, it just eats memory of which there is not much of to begin with. Almost invariably, the site doesn’t respond or you get a server error when the feed update is running.

Number of Posts and Tags

I think the other thing is probably just the sheer number of posts, and probably more importantly, the tags. When I run planet neo (it’s run as a standalone django app, powered by Feedjack), it eats up almost half of the total memory available. I’m not sure there is anything I can do about it other than eliminating the tag portion of the Planet.

GAE to the Rescue?

One of the feeds I am now following is called “GAE Planet“. Not surprisingly, it’s a “planet” running on top of gae, or google app engine. Perhaps, moving Planet NEO to use the GAE planet may be the solution. It’ll get it off of this box which will speed up my other planets. I am a little concerned that it will easily outstrip the “free” usage limitations of gae though. Then again, it may not be a huge deal. May be worth trying on small scale basis to start with…

I’ll need to check whether the gae planet code is available…

I was headed over to Technorati to “fave” hacool. My timing was not good though:

If anyone finds the monster out there, let the folks at technorati know. ;-)

A post by Bill Katz, of bloog fame, posted on the google app engine group a link to Techmeme techcrunch where the pricing for google app engine usage (over the free-of-charge caps) is posted for all to see:

"That cap will continue to apply until later this year, but they will announce the 
following usage fees tomorrow:

    Free quota to get started: 500MB storage and enough CPU and bandwidth for 
about 5 million pageviews per month
    $0.10 — $0.12 per CPU core-hour
    $0.15 — $0.18 per GB-month of storage
    $0.11 — $0.13 per GB outgoing bandwidth
    $0.09 — $0.11 per GB incoming bandwidth

This pricing puts Google App Engine storage and bandwidth costs competitive with 
Amazon S3 (plus Google doesn’t have a per-request fee)."

According to techmeme, the limitation on the number of GAE developers will be lifted and an additional 160,000 will be added tomorrow. Or at least announced tomorrow at Google I/O. That means I’ll be able to register my other google accounts! Nice.

I was creating a presentation for a client using google docs. Basically, I was making a pitch to get him to move to Google Apps. So while I was making the presentation, I went into the control panel to make sure I wasn’t missing any functionality. Well, I was.

Now that I think back on it, I remember looking at it before, but seeing as I use django for my actual website and blog, or plone in the case of neohawk.org, I blew right by it without out really checking it out. Well, today, I finally checked it out — Google Pages.

I know the particular client I was creating the website will not be looking for anything overwhelming difficult, web design wise, so most likely the default templates are just fine. However, the other client will most like want some customization of the templates. It’s not clear how to do that though — at least until I hit the google groups for pages. There was a hint of how to change the css, widen the usable portion of the layout etc. I’ll admit, it’s not elegant, but it seems to work. I did a quick test trying the neohawk.info css stylesheet

Google Page Test

I am personally not interested in them whatsoever, but for this particular client and one other it’s absolutely perfect. They have maybe three pages of content, largely just to say that have a web page. Both clients pretty much work on a face-to-face basis only.

I ran across two GAE blogs this weekend. Both of ‘em look pretty good, and seem to be under rather heavy development. Both are available under open source licenses.

Potlatch

Bloog

Commenting on Potlach is provided by a commenting system developed as a proof of concept for demisauce by the same author. I’m actually fascinated by the idea, but haven’t really spent much time getting my head around it yet. Both of the above are straight up “blogs” as opposed to Monologista or Meow type of twitter micro-blogs.

Speaking of which, I noticed that on themonologista trac, they are looking for developers of client applications. So taking a hint from spaz (for twitter) and the pownce Air client, and realizing that both Air and Flex Builder are available for Linux I decided to give the tutorials a try. I finally installed Eclipse 3.3 (K/Ubuntu version is 3.2, but Flex Builder requires Eclipse 3.3) and started watching the quickstart video in creating a RIA app. It was my intention to follow along with the video, and creating the application. But no go for now.

Took me a while to figure it out, but design view mode is not currently supported by the Flex Builder for Linux alpha(it is alpha after all). But one of the very first things the video tutorial does is to go into this mode to set up the UI for the application. So for now I gave up. I did download the code but haven’t gotten around to running it in Eclipse yet.

One final thing, but on a different note, I saw that you can use google app domains for authentication on a google app engine application. So, for example, neohawk.info is registered as a google app domain. I did not that in the control panel you can click on “advanced services” and add a google app engine application using your own domain. I assumed that you had to use a google account for authentication, so I was assuming that it was a tad bit limited in terms of a closed environment (I use the term “closed” lightly). Being able to authenticate against the neohawk.info domain removes that limitation.

Ah, I’m getting sleepy. I’m not even sure if what I am writing is making any sense. I’ll post it for now, and review it again tomorrow AM.

Good night all.

In the lost version of this blog, I had a post on monologista, a twitter-like clone built in Django. I even had a screenshot and I had begun a simple english how-to. Then, came along GAE and I got distracted.

Monologistaの皆様、ごめんなさいね。しかも、GAE版がある気がつくのも遅かったな。　

So the other day I posted about Meow, a twitter-like clone running on GAE. I thought it was pretty darn cool until about 15 minutes ago. Since Twitter was done (suprise), I decided to go take a look at monologista. I noticed that some one had mentioned a version that ran on GAE, so I headed over to the monologista trac.

Sure enough there was a link to http://monologista.appspot.com/. Sure enough, monologista running on google app engine. As far as I poked around, there isn’t any “following” on the GAE version while there is on the django version. Thinking about it, moew and twig don’t have it either. There must be something about GAE that makes it difficult. <-- you would think that somebody who is "watching" 4 people would realize that monologista does in fact have "following. Dunce. What it doesn't have are the little profile pics that twitter has. Then again, that my be part of twitters problem, eh?

On the trac server there is an API thingie for monologista and you get an API Key when you register for the monologista on gae. In fact, you, as with all GAE applications, login with a google account, but then create a “username” or nickname used for posting. To the right of the posts, there is a little profile box that tells you who you ware watching, etc. In that box, is your API Key.

I haven’t read the documentation yet, so I’m not really sure what you can do with the API Key.

Would be nice if there was a rss feed or something that I could plug into my webpage or blog. Then again, just the existence of monologista on GAE is more than I could have every asked.

I’ve put the link up to this post by Simon Willison on both Ma.gnolia.com and Delicious. Great tips for debugging python. They are additional notes and tips to go along with this presentation on Slideshare.

Now this is cool. Lovely Systems out of Austria is sprinting this weekend to make Zope3 usable on google app engine. They have indicated they will release the code for it as well.

Having been using Zope for many years, I will be watching this development closely.

In my Google Spreadsheet as a Webform post, I casually mentioned “sites”, sort of an internal wiki-like website feature. Well, via the Official Google Blog, the sites feature is now open to everyone. Until now, use of the sites has been limited to users of google apps.

Update: Techcrunch also posts about it

Over the last couple of days, I’ve run across a couple of interesting GAE applications. Eric Moritz is in the process of creating a twitter clone, called Meow, that uses the google app engine as a platform, which one would hope would offer more stability than Twitter. Assuming, of course, that it doesn’t exceed the 500Mb limit or other limits Google has in place for the preview phase. Eric is also the brain, or royal we anyway, behind feedclowd. Feed clowd is a personal aggregator that allows you to pull in your tweets, blog posts feed, delicious and some other web twoy stuff.

I registered and tried out twisternow yesterday, but I’m not really sure what it does really. Well, that’s not completely accurate. I understand what it does, I’m just not sure if I’m interested. It let’s you preset “title”s to your twits, or basically tagging them by topic. I guess I could see that working.

I thought Twig was a nice looking twitter clone. No following other users yet, but I like the look of the site. ;-)

A couple other some folks may be interested in:

appengine-utilities

PyAMF Examples

The GAE SWF Project

There are more over at the GAE Application Gallery.

Over the last day, there have been lots of postings about Google Health. George over at Brewed Fresh Daily asks folks if they would put their health information up on Google.

Being a google fanboy recently, you would think my answer would be yes right out the door. However, I haven’t thought about it too much actually, so I don’t know yet. Okay, so I’m partially lying. I did click on the terms and conditions thingie, but I actually haven’t filled in my profile yet. Reality is I’m poking around more than anything. Like clicking on “Explore Personal Health Services”. I’m more interested in the API than I am in filling out my own profile.

So would you put your info up? Have you already? Let your voice be heard over at BFD.

Ah, neohawk.info is just plain running out of memory. Not that it’s surprising in that Planet NEO has close to 500 feeds and almost 31,000 posts archived, and with over 100 feeds for Neohawk IT and close to 31,000 feeds, this site is not going to respond well at this point. It is too much of a low end server.

Actually, I think it is more likely that due to the shear number of feeds being pulled, particularly for Planet NEO, is that the time I had allotted for it to run is taking much longer. And so the other cron jobs that I run for the other Planets end up running simultaneously. So all of them are running, and it just takes up too much server resources that it can’t respond to http requests in a timely fashion.

To start with, I’m going to have to change the timing of my cron jobs so they don’t overlap. Another thing I may have to do is remove archived posts before a certain date.

Another option is to scrap the Planets all together. Or move them to another server, but I really don’t have the money or time for that.

But I’ll start with changing the timing of the cron jobs I was gonna start with changing the timing of the cron jobs, but everything was so slow, I just dumped all posts before April 1, 2008. Both for Planet NEO and Neohawk IT.

Adobe release the beta version of Flash Player 10. This new version boasts:

• 3D Effects
• Custom Filters and Effect
• Advanced Text Layout
• Enhanced Drawing API
• Visual Performance Improvements

I am pleased with Adobe’s commitment to releasing versions for Linux at the same time as windows and the Mac. I downloaded the beta and it worked on Flock and Firefox. It did not work with Seamonkey or Opera. Not sure why not, though.

Two of my planets appear to be down — Planet NEO and Neohawk IT. They are not actually down, but for some reason not responding. Even the two that are responding take an awfully long time to respond. I haven’t figured out why, nor do I particularly have time right now.

I’ll look at it this weekend. My apologies to to the folks using the Neohawk.Info Planets.

Update The planets are back up. I had to restart services, as it looks like I was running out of memory.

I was playing around with Google Apps this morning. Recently I discovered that you can use your own domain with google apps — google will even register you a domain for $10 for a year. What that means, for example, is that my rbh at neohawk.info email account is now handled by gmail on the backend, I have a calendar (not open to the public yet), a google docs section, a “start page” and a sites page, which is really like a shared wiki for “internal” use.

As you should be aware, Google announced Friend Connect the other day, and being the dutiful google fanboy, I applied to participate in the “preview release”. What I noticed in filling out the form, was that it was “powered by google docs”. And then this morning, via Ma.gnolia was a simple how-to on how to create a webform using google docs’ spreadsheet. So I decided to give it a try.

I recently got an email, most likely because the person in question saw Planet NEO, which was asking to add me as a contact to the company’s media list — the person in questions appears to work for a PR firm. Of course I responded, (queue old lady from Wendy’s commercial) “Where’s the RSS feed”. And that got me to thinking about the number of website’s I’ve seen that have “news” sections on their website, but no RSS feed. And that gave me the idea for my first google docs’ spreadsheet webform. Note, it’s real simple and not really thought out — I did say I was testing yes?

Feel free to give it a shot: Survey. Results are here

I am not impressed with Hardy, the most recent release of kubuntu/ubuntu. First off, you have to go through a lot of hassle(relatively speaking, of course) to keep Firefox 2, the stable release of Firefox. Firefox 2 is the version where Firebug works reliably. However, Hardy comes with Beta 5 of Firefox 3 which is only so-so, and you have to use the beta of firebug 1.1. Since doing installing that though, Firefox consistently hangs after a while. I don’t remember having this much problem with a Mozilla related browser on Linux since Netscape 4.72 back in ‘99.

The other thing that annoys me is that it appears the JRE for Open Office is borked. I haven’t been able to fix it yet. Granted I haven’t spent that much time trying to fix it, but it certainly didn’t work “out-of-the-box”. Now, it is conceivable that since I did an “upgrade” as opposed to a clean install, that may have caused some problems. In fact, now that I think about it, I don’t think I’ve had single Kubuntu “upgrade” go smoothly. Not one.

Okay, so I am probably being too harsh, particularly since overall the experience has been great. And I’m not even remotely considering changing distros, but it is still annoying that it doesn’t just work. I would understand it better if were a “beta” upgrade, but this was a “release”.

This is my first post on my new blog. I have switched to byteflow. I have somethings to tweak, but we should be good to go for the most part.

For some reason, the url is being rewritten to neohawk.info instead of www.neohawk.info. Of course, I can live with that, but that does mean the feeds are all screwy. Actually, now that I think of it, the feed locations have changed anyway, so it’s not such a big deal.

One of the things I like about byteflow over my previous blog incarnation here on neohawk.info is that: 日本語が通るのだ. Yup, Japanese works!

I didn’t bother porting over the posts from my previous blog, as there weren’t a lot of them, and they weren’t anything special either. I’d rather start new and fresh. Actually, I’m more concerned about my Planets as in order to get byteflow to work I needed to upgrade Django to the latest and greatest version. I’m not sure that Feedjack works on the newest version. I’ll guess we’ll see, won’t we.