Eugene's Blog

I can't believe it's blog!

The many benefits of pot

Steve Yegge is one of the bloggers I read almost religiously. His posts are full of insights on the software development topics and in general, and he is always ready with an amusing anecdote from his rich life. His last post Have you ever legalized marijuana? didn’t disappoint either. The author explains “Shit’s Easy Syndrome” (a difficult topic even for seasoned bloggers) in a gay romp that takes a reader from a light-hearted review of a book on “bugs in our mental software” to hapless VPs to many technical difficulties of legalizing a psychoactive drug extracted from the plant Cannabis sativa. But the most entertaining and educating part is the incredible story of the author’s life.

Proving well beyond a reasonable doubt that heavy pot smoking doesn’t affect mental abilities he regales his readers with some relevant events of his life. Apparently while smoking pot he (I am quoting Steve directly):

  • …skipped three grades (3rd, 7th and 8th)
  • …entered high school at age 11
  • …graduated at age 14
  • …had stellar SAT scores
  • …was a U.S. Navy nuclear reactor operator
  • …and much much more!

It is obvious for any impartial observer that smoking pot “on more occasions that [he] can count” at least since the 2nd grade (I assume he was 7 at that time unless he was a wunderkind and started school earlier) didn’t affect his academic nor professional achievements. Probably it even helped!

Implications are mind-boggling. Just think about positive implications for our elementary schools and the nuclear energy industry if we are bold enough to follow Steve’s footsteps.

I can attest that Steve is not unique in this respect. Once I’ve met a guy who skipped not 3 but 4 years (!) — 2001 immediately followed 1996 without anything in between. Later this guy went to space on several occasions (without any protective devices!), and not only met Elvis in the astral plane (a lot of people reported that) but was honored to sing duets with him! I don’t know how he did it — the guy is completely tone-deaf, but facts are facts.

Somehow it circles us back to “bugs in our mental software” and who/what puts them there. You didn’t expect this twist? Good job, Steve!

JavaScript explained

Explaining some dark corners of JavaScript, browsers, or Dojo for the 100th time I realized that I already did it on numerous occasions, and some of my answers are published on public web sites. So I decided to round up the most general ones I posted on StackOverflow and publish links to them here for a future reference.

JavaScript

This section is for language-specific topics. They can be equally applied in any environment.

Prototypal inheritance

A lot of people have problems with the prototypal inheritance in general and its implementation in JavaScript. These are for you guys:

General questions

JavaScript contains some features that are puzzling even for experienced programmers coming from different languages. Some of them are clarified below:

Browser

This section is for browser-specific topics. Let’s face it: many JavaScript questions have nothing to do with the language itself, but with the container of a JavaScript interpreter.

DOM

Browser’s DOM is a source of never-ending bugs, and gotchas. I tried to answer some of them:

General questions

The collection of miscellaneous answers on all browser-related things:

Testing and debugging

The lack of testing and debugging tools is the first thing any newcomer to JavaScript complains about. In reality it is not that bad:

Dojo

Like any library Dojo has its share of misunderstandings, and underdocumented features.

Using Dojo

Hints to help you use Dojo:

Why Dojo?

A legitimate question. Some answers:

Trying out Google Friend Connect

As you may know today Google released Friend Connect. I decided to give it a try. It took about 20 minutes to set up my blog with 4 gadgets: Members, Sign In, Rate/Review, and Wall/Comments. It took about 30 minutes to separate the skin from all these widgets, and set up the Canvas so users can see gadgets in a separate window, if they want to.

Google Friend Connect doesn’t have any server-side components on user’s server — everything is handled by Google. All you have to do is to deploy 2 (two) static HTML files, and add various snippets that can be customized and generated on the Friend Connect’s web site. The resulting snippets are simple, and can be easily understood and restructured.

Now armed with new gadgets I decided to add comments hoping that Google will do the better job of spam filtering than mine implementation. Go ahead and try new features. Sign in (several methods are supported including OpenID!), join my site, rate and comment posts. For now I decided to turn on the moderation of posts to prevent spam and vandals, just in case Google will fail. And remember that it is still the beta, so any problem can happen.

If for some reasons I and Google screwed up the site rendering and it unusable for you, throw me an email. You can find my address on every page of this web site and in the author’s information page.

Using recursion combinators in JavaScript

In the previous post we explored “array extras” and how they can help us to write concise yet performant and clean code. In this post we take a look at generalizing recursive algorithms with recursion combinators — high-level functions that encapsulate all boilerplate code needed to set up the recursion. These functions were added to dojox.lang.functional and will be officially released with Dojo 1.2.

AOP aspect of JavaScript with Dojo

Finally: my open source JavaScript project DCL is built on ideas described in this article. Available for node.js and modern browsers, it implements OOP with mixins and AOP at "class" and object level. Read documentation and background articles on www.dcljs.org, clone its code on github.com/uhop/dcl, and follow @dcl_js.

If we look at the history of computer programming languages, we can see that practically all new programming methodologies were about one thing: taming complexity. The anarchy of earlier days of procedural programming (example: Fortran) gave way to structured programming (Pascal), which was refined with modular programming (Modula), and was reformulated when object-oriented programing went mainstream (C++, and much later Java). And it stopped there. The focus shifted to different branches of computer programming, namely to functional programming, and, to a lesser degree, logical programming. The only major development in this branch was the rise of aspect-oriented programming (AOP) paradigm. Let’s take a look at AOP in our favorite language: JavaScript, and how Dojo helps the language with dojox.lang.aspect package.

Let’s start with basics taking AspectJ as our model AOP language (after all it was designed by the author of AOP). AOP tries to separate and encapsulate concerns, which are common between different modules — so called crosscutting concerns. The goal is simple — it helps to increase the modularity, which is always a Good Thing™.

Google App Engine: the first look

Yesterday Google announced its new offering: Google App Engine. These are my random notes I did yesterday when I studied the new service.

Google didn’t go the same way as Amazon with its AWS. The former offers a form of shared hosting (think “distributed WebFaction”), while the latter offers a virtualized environment (think “distributed SliceHost”). So basically we are talking about more high-level approach to web applications, which is easy even for novices. On the other hand AWS is more flexible and more enterprise-y.

In order to scale, you have to virtualize. In order to virtualize, some actions should be restricted. Google App Engine provides a sandbox on the language level. Only one language is supported at the moment: Python. And guess what? Django is everywhere! For virtualization reasons only the pure Python is supported and a subset of the Python standard library. You can bundle any pure Python modules with your application. Three third-party libraries are available out-of-the-box: Django 0.96.1, WebOb 0.9, and PyYAML 3.05. But fret not, you can bundle a framework of your choice with your application and run it too, even a custom version of Django — basically anything that supports WSGI. I suspect that other mature Python frameworks will be supported out-of-the-box eventually. Pylons and TG2 come immediately to mind.

Google provides its own simplified web framework: the webapp. Apparently it uses Django templates (see this FAQ answer).

Google uses Bigtable as the database backend. Bigtable is not a relational database, so it is not possible to use it with the Django ORM directly. At least not yet. So they provide their own ORM. It looks like … Django ORM with models and whatnot. It is possible to move the existing application to the new database with relatively minor edits provided that you didn’t do any esoteric implementation-dependent hacking in your application. Obviously with Django ORM gone, Django Admin is gone too as all model-based applications in django.contrib. Google App Engine provides its own application for administration, but I didn’t try it yet, so I cannot comment on its quality.

Datastore API supports Django-like models and queries, and Google Query Language (GQL), which looks like a sane subset of SQL’s SELECT statement adapted for Bigtable.

Everything looks simple, standard protocols are used if possible. For example, for distributed applications you have to support some kind of a message delivery system. Amazon does it with Amazon Simple Queue Service. Google uses email messages. Simple yet effective. If you want to connect directly and fetch a file or something, there is URL Fetch API — only http and https ports are supported. Again, simple yet suitable for many things — this is exactly how the internet is glued together.

Another thing you cannot use is Django authentication application. But Google provides a replacement for it, which implements a single sign-on using Google Accounts.

Obviously it is not enough to write an application to host it properly. You should know where you can read local files, and how to serve static files. This stuff is usually done with web server-dependent configs. There is such config in Google App Engine done with YAML. Just like in Django URL paths are implemented with regular expressions. You can designate URLs for dynamic processing. You can define, which directories host static files, you can do it on per file basis too, and you specify how to treat uploaded files. It is possible to specify an expiration time, and set MIME types. Currently there is no way to control compression for static files. All in all you can do most of tweaking described in my article on performance tuning.

I am surprised that they didn’t put on their CDN popular JavaScript/CSS libraries for use in hosted applications. Of course you can always bundle your library of choice with your application, but it doesn’t make much sense for Google to duplicate them. I hope they will host Dojo, jQuery, Prototype/Scriptaculous eventually. Yes, Dojo is already being hosted by AOL, but the Google’s backup wouldn’t hurt either.

Update 5/27/2008: Apparently Google listens: AJAX libraries API (via Ajaxian). Yay!

In real life when you use a database it is not enough to define your models. Most probably you need to set up indexes to speed up frequent queries. Again there is a YAML-based config for that. It can be generated automatically while you debug your application.

Speaking of debugging. The SDK contains the Dev Web Server very similar to what we use to debug Django locally. Additionally it can be used to upload your finished application to Google. BTW, there is no waiting list to download it — you can get it and try it now.

Presently this service is free, but there are some limitations: disk storage is up to 500M, both incoming and outgoing traffic is restricted to 10G per day each, up to 2000 emails per day, and up to 200 million megacycles of CPU per day. Up to 3 applications can be registered per account. All indications are that if your application is below these limits it’ll be free after the trial. The interesting part is yet unannounced payment structure for more resource-hungry applications. Until then it is impossible to do a business comparison between Amazon’s and Google’s offers — they are very different technically.

One of the biggest drawbacks of Amazon AWS is the uptime. Amazon promises 99% uptime, which is not enough. Just imagine that your application can be randomly unavailable 15 minutes a day or almost 2 hours a week potentially at traffic peaks, when you want to serve users the most. Would Google App Engine be more reliable? Only time will tell.

Dojo 1.1 is out

As you all know by now Dojo 1.1 was released in the wild. You can get a lot of useful links and info from James Burke’s post on Dojo 1.1, but let me tell you why you should be excited:

  1. Dojo 1.1 is the first official release, which contains dojox.lang.functional. It was available in the trunk for awhile, but now you can use it without deploying Dojo courtesy of AOL CDN. I am psyched about it!
  2. Improvements in animation, animation helpers for dojox.gfx. A good examples of animation in action is dojo.moj.oe by Pete Higgins (internals were detailed in the blog post, and in the great Dojo Campus article), an example for dojox.gfx animation is the career test. Finally we are getting to the point when we create attractive GUI cross-platform without plugins.
  3. The Dojo API tool. As you know Dojo is essentially a federation of JavaScript modules. Every module can be a library on its own. Now we have a simple way to navigate them, read developer’s documentation without diving into the source, and understand the API. When comments will be added to it shortly, you can add your own notes, and ask pointed questions. Developers are getting an essential tool to see how their modules are documented, what questions are raised most frequently, and so on. This tool will help us to document Dojo even better. All in all Neil Roberts and Tom Trenka did a great job!
  4. Numerous improvements and bug fixes — this is always good. The API is mainly unchanged, so the migration to Dojo 1.1 should be a no-brainer for most users.

I am pleased to see that social aspects of Dojo are getting attention: the redesigned Dojo web site, the Dojo API tool, Dojo Campus is always fun, Dojo forums are active, even the Dojo archive is now easy to navigate!

What’s next? The next target is Dojo 1.2, which will see a lot of enhancements. In my crystal ball I can see myself diligently working on Charting with other volunteers. New graphics improvements and new dojox.lang modules are planned too. While we are busy pushing for 1.2, the maintenance release (Dojo 1.1.1) is planned as well.

On JavaScript, EDP, and 0ms timeouts

What makes JavaScript so different from other languages? Is it its dynamic nature? Its prototype-based funky inheritance? No. The most unusual thing for newcomers is how JavaScript programs handle the workflow. The program looks like a bowl of spaghetti. There is no start or end of the program. What we have here is a bunch of functions, which are called in response to some external events. In most cases we have no way to predict the order of these events. And we know that all callbacks are called from a single thread. Of course we know that it is not a nature of JavaScript but rather a limitation imposed by a specific container of JavaScript programs — web browsers. Majority of JavaScript code is written for browsers and now we have a perception problem. But let’s dig deeper to understand the problem better.

Dojo docs: gfx and dnd

I just finished the first cut of the dojo.dnd 1.1 technical documentation. Let me use this post to remind you about the dojox.gfx 1.1 documentation. I hope I satisfied the curiosity of many developers, but the ultimate proof will be the number of simple questions on Dojo Forum. Let’s hope it will go down. And don’t forget that the Dojo Book has a DnD chapter too.

Both documents are hosted on Google Docs, and are “live” documents — as soon as I update the document it is updated for you too. I intend to update them on regular basis to track the current version.

If you found any mistakes, typos, omissions, or just unclear wording, please let me know. If you know a good trick to do with those packages, do not hesitate to share it on Dojo Campus as an article or a “cookie”.

My next documentation goal is dojox.charting. Stay tuned!

What open source is about

F/OSS is a phenomenon of the modern life. But what makes it successful? Ben Laurie is convinced that Open Source Is Just Economics. I beg to differ. Yes, it does make sense for several companies to gang together and develop something jointly. And they did it for years without the open source. Not always successfully, but they did it. For example, IBM has cooperated with Microsoft on OS/2 back in the days, and there are more examples. What makes OSS so special? Ben’s post has links to other possible reasons. But I think I know the answer: global meritocracy in informal communities — this is the mythical secret ingredient, which holds the whole system together.

If you see that certain piece of OSS sucks hard, you are free to tell it as it is, and suggest the ways to improve it. After all the author of this piece of junk is not your boss, not even a coworker. The informality of this situation gives you more opportunities to be truthful and honest in your assessment. If you are ignored, you can easily fork this sorry piece of code and improve it yourself. Of course you can bitch and moan about a close-source software as well, but if you are sent to hell, you have absolutely no recourse. Yeah, I know, “vote with your money” blah-blah-blah — get real! it doesn’t work in most cases.

If you think that the code in question doesn’t worth salvaging and augmenting, you can start your own competing project. You can do it for a close-source software too, but it is easier for OSS because you can reuse gotchas, certain solutions, and some ideas from your “competitors”.

What if the code is “semi-open” and you can potentially improve it on your own? Example: internal IT software in your company. Well, it rarely happens. The most awful software I ever saw in my life was the internal software of major companies. Internal groups have a virtual monopoly to produce those terrible concoctions. Add a byzantine framework of internal bureaucracy, back-stabbing inter-group politics, and bizarre conflicting agendas of myriad of managers, and you understand why the framework is extremely important. F/OSS thrives on informal communities, which are super-light on bureaucracy and don’t have rigid hierarchies. Suddenly “because I said so” doesn’t work — who are you to tell me what to do? Again I will pay attention to your opinion if you proved yourself in my eyes — this is the meritocracy in action. (No, being a boss is not a sign of your excellent technical skills, sorry.)

So we discussed the “meritocracy” aspect, the “informal communities” aspect, now let’s talk about being “global”. It is kind of obvious: I don’t know a single OSS community, which is restricted by citizenship, or visas, or any other technically irrelevant crap. In most cases the biggest restriction is an informal one: by language. You have to know the language of that community in order to communicate. Some communities can have sub-communities based on languages. It works even for relatively small projects. Example: nginx has thriving Russian-speaking and English-speaking communities with cross pollination.

When you are past the natural barrier of a language, it is all meritocracy again. If a guy from Philippines has the best ideas, and a guy from Poland can implement them well, so be it. Both for-profit companies and Open Source communities recruit from pools of talent, but in the latter case the pool is bigger and more diverse improving the chances to find the right guy for the task.

Successful informal communities create an ecosystem — it is a part of self-organization required to have a sustainable community. The ecosystem includes web sites dedicated to the project, friendly blog posts, mailing lists and newsgroups, and people, who contribute code, answer questions, test, document, and cheer. Nowadays smart businesses attempt to have a symbiotic relationship with OSS projects. And this is the part where economic factors play a huge role. But they came to play with us when the playground was already built for them.