This cleans up the code a lot and removes many ugly hacks. The performance is pretty reasonable and with separate metadata cache it's likely to stay that way.