Tuesday, December 30, 2008

Web User Interface Programming Thoughts

I've been thinking a lot about how we program web applications and user interfaces in general. There seems to be an extraordinary amount of repetitive coding and “boiler plate” code that we just ignore and assume that's the way it has to be. We spend little time doing the actual user-interface design because it takes so much time to just get a simple interface going and write all the supporting code. In my mind something conceptually simple should be simple to implement. For example, I want an ajax type paging grid with server side filtering using some data. Now this is a complex entity if you look at the details of its operation, but we shouldn't have to. We should be able to call a standard API and say “I want a grid here, connected to this data”. This is the path Adobe is going down with Flex and AIR. My question is how we can use similar technologies or develop our own libraries and standards to achieve this same effect?

One problem is in deciding how you abstract your interface through an API. This is further complicated by the layer at which the API works and how a toolkit renders. Let me try to illustrate what I mean with an example. Say I want to put a Date box in my page. Say we're using Dojo for implementation. If we write code using Dojo's API we'll have to follow a certain pattern. In this case, load the Dojo .js file, create our HTML node, and then load up the calendar by programmatically creating the widget or using the dojo.parser. Now what if we decide one day to switch to extJS. Now we have to rewrite all of that code, multiplied by the number of instances where we used it. So the sensible solution is to abstract the Date widget. Since we work primarily with XSL stylesheets maybe we'll create a template stylesheet that we import, than call a date template. Essentially we've just created an API. With this abstraction we have to decide on standard parameters that the implementation will need. How do we refer to the data in the request, we need to pass in the form name or id of some sort. What if we want to disable some dates, now we'll add another parameter. What's the syntax for a date range? Dojo can do this, but will other toolkits? What happens when switching implementations invalidates some of our API abstracting the implementation? Even more troublesome, what if we can't wrap a certain implementation because our API language (XSL) is incapable of processing the parameters and translating them? Even worse, say we want to use Flex, or GWT. They use completely different rendering pipelines that will be nearly impossible to implement with just stylesheets. Now we need to involve the Java layer (for GWT). So this xsl API would really only work for toolkits that can be initiated purely at the HTML layer.

We've revealed a sort of classic programming dilemma, one that I'm sure has a fancy name given by someone who thought about this before me. How do you abstract something as complicated as user interface widgets with an API and not be bitten by implementation eccentricities, limitations of the programming language your API sits in, or major differences in the approaches of different implementations?

If we step back, we'll realize that the problem is in where the API sits. The problem is in how the web rendering pipeline works. The current paradigm touts model-view-controller and separation of data from view. They claim that this separation occurs at the XML > XSLT stage, or even HTML > CSS. But this separation really occurs between the database and your web application programming language (Java in our case). Once you enter your app code, you inevitably affect the rendering and user interface. The app code controls your page state, your cache, your ajax servlets. It's impossible to do anything with ajax and not involve the app code. So your UI toolkit must interact with the app code and this is where your API abstraction must reside if you want to do anything as complicated as ajax.

This seems to indicate that GWT or Flex is really the right way to go. Maybe it's just too difficult to abstract your interface through an in-house API. Maybe you just have to pick a powerful toolkit such as these and be married to the implementation. And as long as these toolkits are designed to be themed and let you extend, and create new widgets, you should be able to achieve nearly everything you could want.

Thursday, October 30, 2008

Defining Marriage

On a more serious note, one of the main issues surrounding gay marriage is a matter of semantics. Some people define marriage as strictly the union of a man and woman. Others interpret a broader meaning.

At first, this seems trivial, why not just broaden your interpretation? If a homosexual couple says: "We're married" then marriage has a broader meaning. The same word can be used in other scenarios as in the example from webster: "seafood marries with other flavors". So why restrict the usage of the legal world?

Well, there does have to be a legal definition somewhere. Otherwise, if the legal word and dictionary word were the same people could go around marrying inanimate objects and whatnot. So marriage does have to be defined somewhere in our legal framework and does need to specify who can marry whom.

So to appease those with a strict marriage definition legislatures created a sort of euphemism: "civil union". But if we have marriages and civil unions and both have the same legal consequences, what legal difference is there between them? The only difference I see is a label. And that label allows government official to discriminate. If both unions have the same legal purpose then they should be the same thing and labeled the same. Either we call unions between people marriages or civil unions.

Gay Marriage and the future of humanity

I've begun to wonder if humanity will ever change. Will we continue to argue over civil rights every time the world changes a little bit?

In the 20th century we argued over women's rights and racial rights. In the 21st we're arguing about homosexual's rights. I'll predict that in the later part of the 21st we'll argue about genetically modified human's rights, in the 22nd we will argue about robot rights and in the 23rd alien rights.

Now you're thinking I'm a crazy person who just wants hot alien robotic sex. Maybe...but I'm trying to make a point. And besides, the issue is not about sex. It's about love & marriage. Otherwise people would be up in arms about blow-up dolls.

In response to something like gay marriage, people against it often bring up the slippery slope argument. They ask what prevents someone from marrying their sister or a horse? Well, the first is actually genetically damaging. But if you weren't to have children one could argue for it, though it seems psychologically disturbing to me. The second (the horse) isn't sentient or able to communicate at the level necessary for meaningful love or marriage.

So, if the requirements for marriage are sentience and the ability to communicate then there's no reason people won't be marrying robots and aliens in the future. And there's no reason homosexuals shouldn't be able to marry now.

Friday, August 15, 2008

BumpTop desktop criticism

There's a desktop project called BumpTop, which aims to make working with your computer files just like working at a real desk. The thought that this is the future of computing interfaces makes me cringe. It's almost as disturbing as the idea of shopping for real items by navigating a virtual avatar through a virtual shopping mall.

Now I'll give BumpTop some credit. It's a good example of how radically different our computing experience could be and how much potential there is for innovation. The project shows how the idea of physical position is significant in the human experience and can be used in user interfaces. Unfortunately, BumpTop just takes the physical analogy to literally to be useful.

One of the most obvious problems with BumpTop demo is the lack of labels. How do you know which PDF is which? If you make a little thumbnail of the content can you really identify the file when it's just a half-inch square on your screen? If you turn on filenames they'll overlap each other and be difficult to read. What if I want to look at additional information such as size, modification date, etc?

Next is physical usability issues. Is simulating a physical environment really that useful? If I toss a file onto a bunch of others do I really want it to move the files it impacted? How will BumpTop represent thousands of files in a directory, won't this be a big mess? How is rotation of icons useful, doesn't it just make my brain work harder to recognize them? What if I can't see something because it is obscured by other files?

I think the greatest and saddest irony of BumpTop is that it aims to improve the problems with the contemporary desktop by replacing it with something better but instead extends the central idea of the current desktop (physicality) to an even more frustrating and extent. Think about it this way:

Imagine we're all using BumpTop. Someone comes along and says "BumpTop is great and all, but it's got some problems: it's annoying when your files get rotated and I always accidentally bump files with each other and sometimes the 3D perspective prevents me from seeing files hidden behind large stacks". Then someone comes along with a solution: "Know what'd be great, a desktop where files can't rotate upside down, and they can't bump each other away, and instead of using stacks, we'll have folders that contain all of the files without obstructing other files on your desktop". Sounds a lot like what we've got doesn't it...

Now I do believe we should have better desktop UI's, but BumpTop just isn't it. I'll elaborate on what I think we should do in another post.

Monday, July 7, 2008

More thoughts on the future of the web

This is a continuation of my earlier post titled It's time for a new Web. I wanted to ramble more on the subject.

So fundamentally what we're talking about is bridging the gap between the internet and your operating system. Letting web applications install libraries that have direct access to graphics, sound, I/O, and File System layers. The trick is exposing all of the important interfaces in a useful and secure way. The question, is how low-level do you go and how much freedom do you allow?

How low?

One idea would be to simply take standards such as OpenGL and OpenAL or maybe SDL and make that your base API. Of course you would need additional APIs for other things such as networking and local file storage. Most programers would be more than happy with this level of access. In fact, most would be using libraries that acted on top of OpenGL and simplified and abstracted even further.

But why stop at the level of OpenGL? What if you just exposed the base hardware layer in a way that OpenGL would just be an internet library on top in the sense I talked about in the previous post.

How much freedom?

The question of how much freedom you give the programmer is important. For example, if you just gave direct access to your local filesystem malicious sites/applications could wreck all kinds of havoc on your system. So in this case it makes the most sense to just have a space allocated to the application that is managed by the browser. This space would be insulated from all other applications. The downside to this is that you would lose the ability to interface directly with other applications data. However, I feel that it's a better design to have the application provide interfaces on it's own with some kind of communication protocol.

Another problem with too much freedom is the loss of structure. I mentioned this earlier as well. If the programmer is just thinking about pixels and the pixels are making up text what tells other applications that the pixels equate to ASCII characters? In other words, how do we apply semantics to different concepts so we can do things like Copy-Paste.

Self-Aware

One critical part of this vision of the web being a success is to have self-awareness. In other words, an API that lets you do things like query the applications and libraries that are installed.

The idea is that the browser doesn't have a defined UI (though there would be a basic default one). Your home page is your desktop environment. So users would chose different home pages such as Google or Yahoo and those pages would provide the user with their Application links, taskbar, tray, etc.

Of course certain interfaces would need to be defined. Such as the concept of an application being open, it's window properties, user messaging (toaster gui), and other basics. However, these would not be concepts with attached graphical standards though they would often be graphical. What I mean is an application wouldn't know how it was being accessed or represented; it would only know if it was visible or not and its size. (maybe some other info as well, but not much) Of course, the application could attach to events and query the other windows if it wanted to interact in some way.

Part of being self-aware is having an event system that all applications could access. So events would be fired when applications are closed, shown, entered, exited, etc. This would let the desktop app do something like track time spent in each app.

Breaking out of the box

While we're at it why not consider implementing things such as P2P as a standard. Wouldn't it be nice if you just downloaded an application update from your coworker next-door instead of the main site? What if P2P was a standard resource for programmers?

What about user interface considerations. Multiple mice, multiple keyboards, multiple monitors/screens. How would all of these interfaces be provided/queried? How do you abstract their input or do you give the raw input and let libraries deal with it? What about new kinds of devices? For example, multi-touch-pressure screens could be abstracted as multiple cursors but they're really a whole array of pressure values. Do you let the browser layer abstract such a device as a cursor or provide its input directly to applications, or both?

Ending thoughts

Hopefully some of these ideas made sense and you understand what I'm imagining. In a sense this is the holy grail, the thing that would remove operating system barriers and completely standardize computer software while letting programmers achieve anything they could imagine. I don't think it would be easy. You'd need to have a good core group designing the specs and then a marketing engine that could convince people to develop with the new standard and gradually bring it to the mainstream.

I think if such a system were to arrive it would first be like today's browsers. It would have it's own HTML rendering engine and act just like any other browser with the exception that it could do so much more. As developers began releasing internet applications through it and converting existing ones people would live in their browser more and more. There would be a point where it seemed like you had two desktops. (you already have two taskbars with tabs in your browser) People would begin to fullscreen the browser most of the time; eventually operating system distributions would focus solely on getting the user loaded in the browser and not even bother with its own desktop environment.

Tuesday, July 1, 2008

Thoughts on Web Knowledge

Forums suck. Email archives suck. In fact, pretty much all the web content you get from a google search for the solution to a problem sucks.

But let me start at the beginning...

I recently got a second monitor and got in running in Ubuntu Linux. I'm running an Nvidia card in a twinview configuration. I had one major frustration: Windows would maximize across both monitors and dialogs would appear right in the middle of the virtual screen across both monitors instead of within the middle of one.

Today I decided to find a solution to these issues. So I went to Google and did a number of searches.

I dug through a number of forums and email archives looking for the solution. I finally found half the solution on a blog and the other half in one of the forums. (I ended up having to remove xserver-xgl and adjust compiz settings)

Anyway, the point is that across the internet there's a huge duplication of knowledge and it's incredibly unorganized. Also, it is often misleading, out-dated, and unhelpful.

The biggest culprit is forums. Forums are poor organizers of knowledge. Threads are often duplicates and experts are often dubious. And sometimes the gold nugget is a comment 3 pages into a thread surrounded by 2 idiotic posts.

So what's the solution?

I'm thinking something like wikipedia, but directed at practical knowledge. And not a wiki per se, but another type of user submitted content system with more structure and a better sense of context as well as dimensions such as time.

It's Time for a New Web

The visual web is being pushed to its limits. What developers call Web 2.0 is really just a creative use of Javascript and a maturing of web design practices, which are mostly a result of better server side technologies and development techniques. Unfortunately, behind the scenes of every "Web 2.0" site is some pretty horrid technology layers that are being used in ways they were never designed for.

Many of the layers and interfaces in a site don't stack perfectly and have cross-cutting concerns that are hard to reconcile. For example, with CSS and HTML the idea is to separate your presentation layer from your structure. However, there is a significant amount of cross-over. For example, order of content, which is presentation, is largely defined in HTML. (the order of columns in a table, the order of sections of your page, etc.) You can absolutely position divs in your page with CSS, but you are limited in how you do so.

All of this aside, there are major rendering issues with HTML. First is the lack of pixel perfect rendering and browser inconsistencies, which add significant frustration and development time for designers. The next issue is that HTML was designed for static content. Animating and dynamically changing large portions of the page, as the new sites do, appears choppy at best and may have visual flaws.

The success story


HTML has had enormous success as the premiere communication medium on the web. I just wanted to highlight its virtues and explain why I think it has been so successful.


  • Hyperlinks: The defining concept of the web is the hyperlink - the idea that one HTML page can point to another

  • Powerful/Extensible: As we've seen recently, with some clever use of CSS and Javascript you can do some pretty fancy things. With plugins such as flash and java applets you can do even more

  • Human Readable, Low barrier of entry: You can pop open a text editor, type in some HTML and view it. You can copy this file to a server and serve it

  • Open Standard: The fact that HTML is a standard that anyone has the right to implement a browser for is essential to its acceptance


The Essence of HTML


I want to look at the fundamental technological essence of HTML. Basically, you have a Client and a Server. The Server has some content, the Client wants to see it. HTML is the medium or language you use to describe the content. There are many hidden aspects to it, such as how a hyperlink works, what meta-data a site has, asynchronous communication, etc. However, I want to focus on the media aspect of HTML. (Audio-Visual)

There are two essential aspects to HTML as a media.

1. It's like a compression algorithm

If I want the client to see some text I could send it as an image. But it's far more effective to just send the ASCII characters and have the client's font engine render the text. Essentially, this is compression. I give a few parameters: the text, the font metrics (size, weight, family, etc.) and at the end I get my rendered text.

2. It has semantic meaning

Certain concepts within HTML gain semantic meaning within the client's computing environment. For example, screen readers can read the text. If I copy HTML into my word processor, the word processor translates it into it's own markup. I can also copy things like tables into my spread sheet program. The same thing goes for things like images. This interface to the rest of the user's computing environment is incredibly useful.

A blank canvas


So lets start talking about a new web medium. If we could start again, if we had a blank slate, what would we want, how would we design things differently?

From an idealistic point of view we want a blank slate, complete technological freedom. Just give me a accelerated 3D canvas with a primitive drawing API and I'll build everything on that foundation.

But why limit it to graphical freedom? Why not have complete freedom?

Imagine your browser had a library versioning system. When you visit a new site the page defines the libraries it needs and where they can be found. Any libraries a client does not have, it downloads. There would be a number of standard libraries most sites would use: font engines, image format rendering, etc. Imagine the possibilities: You could provide new image formats, new font engines and fonts, on the fly.

On the other hand, this would be complete technological anarchy. What's to limit the number of libraries? What are the security implications?

There's also the whole side issue of maintaining semantics within the web. However, I feel like this is actually fairly straightforward. It would be each library's responsibility to attach semantic data. The font engine would attack text semantics to its rendering. The image library would do the same.

The New Web



If this could be achieved you would have completely redefined the internet and desktop computing experience. Instead of firing up an application, you would go to a certain url. If done right such a standard would erase operating system barriers and become the defacto standard for software distribution.

You might say this sounds a lot like Java Web Start. I would say the goals are similar but the key is that such a system would be language agnostic. Not only language agnostic but future language compatible.

Let me go over a detailed example. Imagine I wrote a new scripting language called Ascript. I write its interpreter in C and compile it for a few different processors and package it according to some web library standards. I then write an application in Ascript. My application has a standard web header, which specifies the Ascript dependency. I put this application on my webserver. When I client visits the url, they download Ascript if they don't have it, then the browser launches my Ascript script with the downloaded interpreter.

There are a lot of issues to resolve at this point. What would the API be like? Would you have a direct interface to OpenGL? How would the user control application access? Maybe have dialogs such as "X wants to access your file system. Approve/Deny" How would the distribution work? Could you assist distribution with automated Bit Torrents? How would applications be organized? Would they have desktop/start menu icons? What other APIs are available? Where do you draw the line? Could you execute assembly?

I'll probably elaborate more on these ideas in the future.

Thursday, June 12, 2008

Synchronizing your computer

Once you begin interacting with multiple computers you inevitably start duplicating a lot of work and things fall out of sync. You have two sets of contacts, two sets of bookmarks, duplicate files that have been independently worked on, etc. This becomes incredibly frustrating and unproductive.

One solution these days is to do all your work on the web. For example, write all documents in Google Docs, use your email web application's contacts system exclusively, etc. This way all of your work is stored in one location (on the web application's servers) and presumably backed up by the web service.

But this doesn't work in all cases, and inevitably there will be files you want to synchronize between computers.

I only have a small number of files I want to synchronize:

  • Personal Contacts (Thunderbird)

  • Bookmarks (Firefox)

  • Personal passwords in a password safe file.

  • Notes/TODO lists, often using Freemind

  • Other files I want to access from work & home


USB Stick


One solution would be to carry around a USB stick with you all of the time and always work off it or use something like rsync to synchronize the files with it.

This wasn't going to work for me as I'd inevitably lose it one day and be screwed...

Setup a server


I thought about turning some old machine into a server and using Subversion or something similar to synchronize my files. However, managing a server is a pain. Properly setting up backup scripts and keeping the machine running is a lot of work that is better handled by a professional company.

Online file backup/synchronization solutions


This is the best solution in my opinion. You basically download a little program that sits in your taskbar and synchronizes your files with the company's servers whenever you make changes to the files. My main criteria were that it be cross platform, synchronize easily, and have good performance and design.

Unfortunately, these conditions eliminated a lot of candidates.

I ran across a slick little web app called DropBoks
This lets you just upload and download files but you have to do a lot of clicking. What I really want is to just point some software at a folder and say synchronize this.

The next one I looked at was Power Folder. I tried the 30 day trial and it worked sort of...but the UI was terrible and there were performance issues. My roommates complained of lag spikes when I had it running.

After more searching I found Spider Oak.

Spider Oak seemed to be everything I was looking for. A clean UI, good performance, a progressive development team. It's even free unless you need more than 2 gigs. However, in its current state, Spider Oak doesn't support 2-way synchronization. It only backs up a folder you select and makes it available for download on other machines, keeping those folders up to sync. It does not update changes to the folder on the second machine back on the first. I contacted the developers and they responded with:

As of this moment, we do not allow the true 'syncing' of folders across computers. However, we will be releasing this functionality within the next couple of months at the latest. That said, you can easily view and download data you have stored on any number of machines from any one of the machines. You can accomplish this by downloading the SpiderOak software on other machines you want to back up. Once complete, you will see the other machines on your SpiderOak Network on the 'View' tab. If you then click on another device than the one you are on, you will be able to browse all of the data on that machine as well as download it.


So I'll just have to wait until they implement the feature.

Firefox and Thunderbird


I use a couple Firefox/Thunderbird addons for Contacts and Bookmarks.

Addressbooks Synchronizer uses IMAP to synchronize your address books.

Foxmarks is a free service that synchronizes your bookmarks online.

I have a blog!

Well, I've decided to try blogging...it was bound to happen sooner or later.

Why "the fortified hill"? Well, apparently that's what my name (Arlo) means. I guess it applies to me in that I can be stubborn and unmoving, that I don't easily change. But then again, most people don't.

I'll start filling this blog with some more substantial posts soon.