Oct 30 2011

My Asus EEE has recently started acting up (I believe there might be some electrical problems), so I was on the market for a new laptop. I’m the kind of person to carry a laptop everywhere so the netbook concept really appealed to me — I honestly don’t use a computer for much more than coding, email, music, and web browsing, and keeping things lightweight helps a lot. I was looking for an Asus since I’ve had good experiences with them in the past and was pleased that the EEE came in a Linux-only version (driver support on the EEE was remarkably good for a laptop). Luckily the Zenbooks became available right as I started looking around and the hardware (specifically the UX21E) fit my needs.

While the Zenbooks are decent computers, Linux’s support for them is mediocre right now. This is due to two major factors: Sandy Bridge has a graphical architecture that slower distros don’t have support for yet, and Asus selected some hardware components that have poor support right now. Luckily, there seem to be few obstacles to a good user experience other than driver maturity, so someone looking to purchase a Zenbook to run an alternative OS may have better luck in a few months.

Here are a list of issues I’ve encountered, with solutions where I’ve found them:

• Working drivers for the ath9k chipset are available in the latest kernels, but not in the stock Debian Squeeze release. If you are installing Debian, I recommended doing a non net-install with a ‘standard’ ISO, transferring the sources for a newer kernel, building it, and switching to network repositories once wireless is working. The Arch Linux installer has working wireless drivers and I assume Ubuntu would as well. If you find yourself stuck in Debian without NetworkManager, you can bring up a network following the Arch installer instructions for set up wireless in the live environment (make sure you install wpa_supplicant from the ISO).

• The RFKill module has 3 on/off endpoints: PHY, Wireless, and Bluetooth. You will need to manually enable PHY in order to use wireless at all, I have a script which just enables everything at boot. I’ve put that up on a gist.

• There are some battery tweaks that suggest enabling ‘min_power’ for SATA power management. I saw errors pop up on dmesg when I enabled this; I can’t tell if they’re harmless or not but I felt it was prudent to leave it off.

• ACPI support and sensor information is limited. I had to manually place `acpi-cpufreq` and `coretemp` in `/etc/modules`, and the only readings I get are temperature. Battery discharge rate is available with `acpitool -B` but not in `gnome-power-manager`. This means one can get battery percentage estimates, but no time-till-empty estimates.

• The hotkey (function key) driver generally works OK (you can disable wireless, control volume etc.) but it has one annoying quirk: you can’t type an open parenthesis. To do so, you must type Fn-Shift-9 instead of simply Shift-9. I hope there is a tweak I can make to correct this somewhere.

• The trackpad is from a new, difficult to correctly spell manufacture called ‘Sentelic.’ The hardware supports two-finger scroll and multitouch, and the drivers (at least on 3.0.4) claim to as well, but somehow these don’t work when enabled. These appear to be driver-only issues, especially since they report ‘unexpected absolute data’ which is ignored. I can only assume these are the scroll events and simply aren’t handled yet. A much larger issue is the inability to (even temporarily) disable tap-to-click: these goes off while you are typing and makes it virtually impossible to correctly input text. There is a shell-based daemon someone wrote that I use now, I’ve modified it to use xinput to disable mouse input completely. It has a few issues, but I hope some driver enhancements come out to solve the problem more completely. In the meantime I’ve written a daemon to handle this much more smoothly.

• Graphics support, at least on Debian, is not good. Sandy Bridge has a new acceleration architecture which is available in newer Xorg setups, but not in the ones shipped with Squeeze. I may upgrade to Wheezy or put time in to install Arch just to get this. The most notable effects are comically slow screen refreshes and an inability to run dual-head. Again, if you use a newer distribution these issues may be resolved.

• Attempting to suspend the laptop results in a crash. I suspect this is related to the outdated drivers and perhaps a newer distro won’t see this issue. Luckily the laptop both boots and shuts down very quickly, but it is still very inconvenient.

• By default, the speakers are muted. Bring them up with `alsamixer` :).

Sep 21 2011

Here are some more Android apps which I’ve been using:

• Twicca (QRCode) The ultimate twitter client. Beautiful interface, perfect posting/notification setup, awesome broken English “retweeted by 10 peoples,” a number of plugins for image upload, Google+ reposting, etc. Highly, highly recommended.
• Pixel Art Clock Widget (QRCode) iOS guys always ask me how many clocks I have (“Android is all about clocks”) and I answer: 4. This is one of them, and it’s great (especially if you disable the illustrations).
• Aix Weather Widget (QRCode) A weather widget that tells you exactly what you need to know: what the temperature is like in the next 24 hours and if it will rain. This is the first weather widget I’ve found useful; I hope the developer adds more features (and data sources) as time goes on.
• MyChain (QRCode) An app designed to force you to keep recurring habits by “not breaking the chain.” I have an entire home screen dedicated to these widgets, and it seems to work OK.
• Meditation Helper (QRCode) Barebones meditation timer that can chime bells at specified times. Keeps a sitting log and gives you a MyChain-like widget to keep track of your progress.
• Widget Timer (QRCode) One-click countdown timer widget.
• Simple Stopwatch (QRCode) One-click stopwatch widget.
• HexDefense (QRCode) A great Android clone of geoDefense. Fast-paced tower defense that feels like a combination of tower defense and RTS.
• AirPush Detector (QRCode) Did you get a pop up ad in your notification bar from these fuckers? AirPush is a fantastically douchey way to display ads on Android and is taking all the fun of pop-ups to your mobile device. The best part is you rarely know which app uses this shit! Luckily Dan Bjorge wrote this tool to tell you which apps have AirPush enabled. My recommendation: uninstall these apps and give them 1 star. Don’t “opt-out” of AirPush, developers need to know that using it will cost them users.
• Android Agenda Widget (QRCode) My favorite calendar app. Insanely customizable, lets you view/edit/add events right from the screen. Integrates with Astrid and other to-do apps to put your to-do deadlines right on your calendar. I have the plus version and I highly recommend it.

Sep 15 2011

I tend to be pretty eager about trying out new and faster filesystems, so when I got the opportunity to start using btrfs on my EEE I set it up as my main FS. Unfortunately, performance started to suffer — over time I began to see multi-second spikes in latency, massive I/O pressure for simple commands, and all kinds of system weirdness. Using Chrome slowly became unbearable and I avoided opening GMail whenever possible — opening GMail and Twitter simultaneously nuked the browser. I blamed Chrome and JS-heavy websites, but it turns out btrfs was to blame, as evidenced by this post. If you are runnint btrfs and are hitting these problems, upgrading to a 3.0 kernel and mounting with “autodefrag” should help — while still being tested, it helped on my system. If you’re thinking of switching, I’d recommend sticking with ext4 until performance stabilizes a bit.

Aug 21 2011

I recently started on a small ruby CLI application named currently. I had heard about a testing framework called cucumber, and when I stumbled across a CLI extension called aruba I got pretty excited. Could this be a fully featured, well-kept version of my own rspec everywhere? The answer is yes. And no.

While `aruba` does a pretty good job of handling a lot of CLI app testing cases, the `cucumber` language itself is piss-poor. Here is an example.

These are `aruba`-specific, but I think you get the idea: `the exit status should be` is a language construct. You must type it that way. The capitals at the beginning of ‘sentences?’ Language construct. I feel physically gross typing this out.

This whole thing reminds me of another language which was targeted toward a ‘business-readable domain-specific language’: COBOL. This is not a coincidence, the main `cucumber` page links to a page attempting to resurrect this idea. The problem with these kinds of approaches is they patch up a superficial part of programming and ignore the real things that are hard. Learning semicolons and parenthesis isn’t the hard part. Programming is hard because the computer never does what you think it’s doing, and it’s not as simple as trying to slather on an “English” layer.

Our brains don’t have the ability to follow all that goes on in a computer simultaneously. Thus, we have a poor mental model matched to a fast machine that can outfox our mental model very easily. This is the root of the difficulty in programming. All bugs flow from this cognitive impedance mismatch, and any language or methodology which doesn’t address this will be mediocre at best or actively detrimental at worst. Admittedly, I started learning programming a long time ago and can’t place myself in that kind of mindset now — perhaps the English approximations are useful for someone grasping the workings of the machine. But I fail to see why an experienced developer would create a tool for other experienced developers with this methodology selected.

Aug 9 2011

I recently got an HTC Thunderbolt (Verizon) and wanted to catalog a few of the apps I have so far (in no particular order). Note that I am leaving out the obvious ones like Google Music/Goggles/Docs/anything, Skype, etc.

• AppBrain (QRCode) Keeps a database of all the apps on your phone and lets you look for similar apps, recommendations etc. Also lets you queue installs/uninstalls from the website, and keeps track of app statistics (updates, price increases, etc).
• Overlook Whiz (QRCode) A widget that you can put on your home page that will ping a server to make sure it’s online.
• Overlook Fing (QRCode) Gives you a nice total view of devices on your network, along with some ping/scan tools.
• Battery Monitor Widget (QRCode) Really great battery monitor program, shows current charge/usage in mA and computes battery time left based on averages and instantaneous use. Developer releases updates frequently. Can calibrate itself with a full discharge/charge cycle (my battery was reporting 1350mAh but actually is around 1450). Can also provide a percentage in your notification bar which is way more accurate and useful than the stock battery bar (note: this may be only in the paid version).
• JuiceDefender (QRCode) Automatically enable/disable mobile data and wifi depending on usage (and, with the paid version, location). Occasionally may interfere with constantly updating apps but with the right configuration it should be OK. Some people think it’s snakeoil but Battery Monitor Widget has been showing a roughly 25% increase in battery life for me. The nice thing about JuiceDefender is it’s relatively automatic; other than setting up some rogue apps that don’t sync correctly, JuiceDefender is roughly fire-and-forget.
• System Tuner (QRCode) A companion app to BMW (above), written by the same guys. System Tuner is the best process manager/CPU monitor/task killer I’ve seen so far. It also can place a CPU usage counter in your notification bar. Apparently you can link it in to BMW to see which apps are sucking the most power but I haven’t figured this out yet.
• G-Stomper (QRCode) Very fun and straightforward drum machine/groovebox. Easier to get up-and-running with something cool on this than other ones I tried.
• Rain Sounds (QRCode) A collection of rain loops; there is an app with the same name that isn’t as good.
• Kalør Clock (QRCode) A very classy nightstand/desk clock. I use it as a desk clock while my phone is charging (the Thunderbolt has a kickstand and terrible battery life). The devs are cool guys and responsive to new features/suggestions. I wish they used a different weather API than Yahoo’s, that tends to suck. Combo bonus: Danish.
• Campyre (QRCode) Native Android client for 37signals’ Campfire
• Digitally Imported Radio (QRCode) Streaming electronic music
• Gh4a (QRCode) A native GitHub client that shows you activity on repos you’re following.
• Pixel Rain Live Wallpaper (QRCode) Nice live wallpaper, I personally have it set to a few drops, no shatter, slow fall, long tail. With this you end up with streaks that crawl down your phone and look very… computational.
• GO Weather (QRCode) Accurate multi-city weather app. As far as I can tell it uses wunderground data which tends to be hella accurate. Combo bonus: Chinese.
• Sleep As An Droid (QRCode) A great alarm clock that attempts to wake you during times of light sleep. Has some nice features others don’t such as automatic recording of snoring and a life changing ability to notify you when you should go to sleep. Auto-calibrates itself so make sure you use the “delay tracking” feature to only start tracking when you’re actually in bed. Very happy with this guy, note that you should leave your phone plugged in while it’s running as it can be a battery hog. Combo bonus: Czech.
• Google Authenticator (QRCode) Google’s time-based token generator. Nominally for 2-factor verification of Google Apps, but supposedly the server-side implementation is open source. This means you can lock down any site with 2-factor auth.
• I.O.U. (QRCode) An app for keeping track of who owes whom, including stuff (instead of just buxx).
• Aldiko Book Reader (QRCode) Decent PDF/ebook reader. Gets out of your way, but I wish it kept zoom settings across pages for reading LaTeX-created PDFs.
• Some colorscheme apps:
  • Colorz (QRCode) Nice for picking color-wheel based schemes
  • Color Pal (QRCode) Pulls popular color schemes from COLOURlovers
  • Magic Color Picker (QRCode) Nice touch-based color wheel
• Keygenjukebox (QRCode) Giant database of keygen/crack/demoscene music.
• Prey (QRCode) Open source phone anti-theft tracker. Also works for laptops (Win/Mac/Linux), so it’s a nice solution overall. Has free and paid plans.
• Mint.com (QRCode) Native app for Mint which lets you track your cashflow/transactions/budgets on the go.
• Catch Notes (QRCode) Straightforward note-taking app. Nice to leave as a widget somewhere so you can quickly stash something.
• Swype Keyboard (QRCode) Crazy good swipe-based keyboard. Currently in some weird free beta thing.
• GPS Status & Toolbox (QRCode) Really nice GPS display/monitor app. Also integrates accelerometer to show you roll and tilt as well as GPS satellite info. Lets you display the compass direction of locations in “radar” mode to finally achieve the videogame-style waypoint navigation I’ve wanted from a GPS forever :).
• Astrid Todo List (QRCode) A very well designed to-do list for Android; one of the two life-changing apps I’ve purchased (along with Sleep As An Droid). Astrid supports random reminders, location-based plugins for Locale/Tasker, syncing with the Astrid website and Google Tasks (although the latter is flaky at the moment). Buying the Astrid power-pack gives you a full-sized widget that you can put on your home screen to have handy.
• Tasker (QRCode) Very full-featured profile-based phone control system. Turn your notifications down when you’re at work. Sound a chime when you get home. Keep the screen on when Kalør Clock is running. Hook into Astrid with the Locale plugin to only show tasks at certain times/locations. Tasker has a plethora of options and actions; it’s a bit overwhelming to use (and the UI is non-intuitive) but it’s pretty damn powerful.
• TimeDroid (QRCode) Time tracker that uploads directly to Freshbooks. Simple and effective.
• My Tracks (QRCode) Location/route tracking app, good for timing commutes
• BART Usher (QRCode) A barebones BART app that loads data from BART’s API. It would be nice to have a nicer interface or some more real-time data, but as it is the app is useful to look up times.
• RunKeeper (QRCode) Fitness tracking app, designed for running but can also be used for cycling, hiking etc. Can upload to the RunKeeper website and keep track of calories etc.
• Satellite AR (QRCode) Shows you which satellites are in the sky and where.
• Nanoloop (QRCode) Small, minimal synthesizer based on a GameBoy chip. Very fun and absorbing, you can spend a while making sequences here.
• RPNCalc (QRCode) RPN calculator with a straightforward interface. Easy to pick up and use.

Aug 6 2011

I’ve been using awesome for a while now as my primary window manager. I prefer tiled window managers because they let me think about one less thing (where to place windows and dragging them around) and let me get my work done better. However, I think it can go one step further: I want it to automatically manage my workspaces in addition to the layout of my windows.

My idea for a Lua script (awesome uses Lua internally) would be something that optimizes over two parameters: screen real estate and workspace switching. I want it to automatically figure out “contexts” and only put relevant windows on screen when I switch into those contexts. For example, I often have a ‘coding’ workspace set up, a ‘research’ workspace, an ‘IM’ workspace and a ‘music’ workspace. I would like awesome to, as I spawn and close windows, figure these schemes out and dynamically show and hide windows. For example, I may be coding and need to reference a webpage for a design or syntax example. This script should detect that I keep shifting over to look at the browser and stick it with my code. Later, when I’m simply back in my text editor, the browser should fall back to a secondary workspace to let met focus on just the code.

By optimizing over abstract qualities (“focus” and “minimal context switches”) you open the door to a lot of clever algorithms to actually help HCI instead of hinder it (I’m looking at you, Clippy).

Aug 6 2011

I have an SSD-equipped Asus Eee that I use as my main laptop. Unfortunately the Phison SSDs shipped with the Eee suffer from staggeringly low random write performance. Under load latencyTOP would show several seconds (!) of latency. Some of these issues had cleared up when I upgraded to 2.6.38, but there were a few spikes here and there.

Previously I had been using the deadline IO scheduler because it tends to have increased performance with respect to CFQ on SSDs. I didn’t want to use noop because I actually was optimizing the system for latency; while noop tends to have greater throughput, I was optimizing for latency. For aggregate latency, deadline should be the best performing scheduler on SSDs. However, in my case I don’t care about aggregate latency, I only care about the latency of programs I actually use. This is where both deadline and noop fail.

Strangely, for this kind of service we want to return to CFQ. Not only does it try to schedule based on nice levels, but it also can schedule based on classes of traffic specified by ionice. For example, my backup software tends to generate a lot of read/write traffic in the background. While the reads do little harm, the writes tax my poor SSD and cause the multi-second latencies described above. This has been partially documented here. To help alleviate the problem, I can run SpiderOak like so:

ionice -c3 nice SpiderOak

This sets the application to have low CPU and I/O priority. The -c3 means the ‘idle’ class; its I/O operations won’t run unless nothing else is using the system. While I have no hard data, I have noticed a noticable drop in latency for other applications with this scheme set up. Note that noop and deadline fundamentally can’t support this kind of I/O scheduling; also note that this advice only applies for SSDs which have poor write performance: cheap laptop and phone SSDs in particular. Most desktop SSDs have significant caching and TRIM support which boost random write speed high enough to remove most concerns. With these drives, using noop or deadline makes a lot of sense.

Most distributions have CFQ as the default IO scheduler. If you’re not sure, you can run this to check:

cat /sys/block/sda/queue/scheduler

To set things to CFQ run:

sudo -i
echo cfq > /sys/block/sda/queue/scheduler
echo 0 > /sys/block/sda/queue/rotational

Obviously adjust sda to correspond the drive you are adjusting settings for. The second line hints to the scheduler that you are using an SSD (instead of a rotational drive) and that it should not attempt to “anticipate” IO coming in on the same drive track.

Jul 13 2011

Firing up top or free on a Linux system and attempting to discern memory usage statistics can be a daunting task for the uninitiated. While the task of explaining “free” vs “used” memory has been done in many places, I wanted to take some time to talk about a little-known tool called Exmap which gives you a much deeper insight into the memory usage of your processes.

I have a netbook with a limited amount of ram and an SSD for a backing store. Since this particular SSD has poor write performance and I don’t want to wear it out with excessive writes, I’ve set the VM swappiness to zero. This means I truly am constrained by the physical memory of the system as I can’t rely on swap to mitigate app bloat. A similar situation arises with some VPS providers which simply disable swap altogether — the 256 megs you get is it. In these kinds of situations, knowing precisely which applications are hogging ram is crucial.

Of course, the copy-on-write nature of the VM subsystem complicates this calculation. While one-off applications are easy to spot in top or ps listings, complex GUI applications load so many common libraries that it’s tough to tell how much memory would truly be saved by killing off an application. This is where Exmap shines.

Exmap scans pages assigned to each running binary and identifies how much ram each process individually consumes, taking into account shared libraries. For example, it turns out Pidgin is an offender by loading many shared objects unique to itself (part of this is its plugin system). Exmap is, as far as I can tell, the only tool which gives you this level of granularity in tracing down application memory usage.

To operate, Exmap needs intimate access to your application’s memory — this means a compiled kernel extension. Unfortunately, the source located on the project’s homepage does not compile directly on modern systems (Debian Squeeze at the time of this writing) and some modifications were needed. The author has a fork on GitHub but this too suffers from some minor compilation warts. The version I have in my repository seems to compile, load, and run OK. The easiest way to get it is by running this:

git clone git://github.com/wickedchicken/exmap.git

and following instructions in the README.

Jul 13 2011

A large part of my current project involves tracing program execution through memory. While there are a few methods of obtaining this data, the most straightforward is with the Valgrind toolkit. Valgrind

1. loads an arbitrary executable
2. translates it into an internal IR called VEX
3. lets you manipulate the code in converted form
4. compiles that code back to x86
5. runs the result

If you do any kind of online program analysis, this kind of framework makes your life much easier. Given the large scope of Valgrind’s operation (live translation of running code is no easy task), I expected the source to reflect the complexity of the problem. Fortunately, Valgrind’s Lackey example tool provides an easy starting point and Valgrind overall seems to succeed at providing a simple but flexible interface.

The first place to start is VEX itself. Viewing libvex.h and libvex_ir.h in the valgrind/VEX/pub/ directory should give you a basic idea of how VEX is structured. Since Valgrind was originally built to support Memcheck, VEX is especially geared toward tracking memory operations; loads and stores have extra emphasis.

VEX code consists of “superblocks,” sections of code with single entry and multiple exits. This corresponds to a glued-together set of basic blocks a compiler might work with. Since my trace requires true basic blocks, I treat an entry-exit pair as a unique basic block.

A Valgrind tool receives each set of superblocks in IR format, behind the scenes Valgrind takes care of decompilation and compilation. All your tool sees is a stream of VEX IR. For speed, the Valgrind JIT compiler caches and inlines sets of superblocks as they are run; if you need thus functionality turned off you can enable --vex-guest-chase-thresh=0. This supposedly turns off the multiple-exit feature of superblocks and makes them true basic blocks, but I have yet to test this in actuality.

The fact that both instrumentation code and instrumented code run simultaneously opens up a host of online analyses previously untenable with compile-time transformations. The closest modern analog would be LLVM passes; I plan to look into these for more performant tests but the compile-time restriction certainly puts it behind in flexibility.

Jul 10 2011

I’ve updated the outer haven / qrunknet servers and installed a fresh whisper install. I’m pleased with the result, the design is modified from Adam Wiggin’s template and switched up with some colors from color scheme designer.