Development Notes

Quoth The Donald... nevermore.

2011-07-18T06:23:00.000-07:00

The Quote

In a somewhat famous statement, Donald Knuth (in a quote he later tried to pawn off on Tony Hoare) allegedly said: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil".

In many years of software development, many people have justified their poorly written code with: "premature optimization" is bad and should never be done because Donald Knuth said so. Very often this was a response when their product fell over under a mild load.

As someone who is involved in code optimization (and have been doing it for over a decade), I can tell you what Donald meant was: to not waste time on making well written code optimally fast and concentrate getting it working well and deal with unexpected inefficiencies later. Profiling and tuning well written code is relatively easy, poorly written code presents a time consuming problem as basic need to be fixed first.

He did not mean that you can just write poorly optimized code and try to justify your actions with a misunderstood quote. Maybe Donald realized the can of worms he had created and tried to distance himself from it; we will never know.

How it all begins

No one writes inefficient code willingly or knowingly. I have never met a developer who had bad intentions. What I have met are developers who have convinced themselves that their coding skills are amazing and they write amazing code. That they were a lot better than they really were.

Case 1

I worked with a developer once who told me that he "was something of a big deal around here", this is before his code was unable to scale past 5 concurrent users and eventually required a complete rewrite (but not before his options vested, he made a lot of money and left to start another company; never learning how average of a developer he was). He was not a bad guy, an average developer at best, but he had convinced himself that he was way above average; a legend in his own mind. Everyone praise him (mostly non technical people). Since he was an average developer who thought he was amazing, he hired people for his group that he thought were average (when in fact they were barely able to write code). The codebase reflected it. When I looked through the code, I saw many basic problems (CS101 level issues). Recursion without stopping conditions through all code paths, this caused an occasional race condition on the browser. Many cases of incorrect collections (like lists of name/value pairs doing O(N) lookup when a map/tree with O(N log N) would have been far more efficient). Places where he used loops inside loops which resulted in exponential slowdowns (single loops were needed). When I asked him about his many basic problems? Quoth The Donald... nevermore.

Case 2

Another lead developer I worked with was constantly told, by people who did not know how to write code, of how he was a superior developer and he had the mentality that he could do no wrong. He spent his development time trying out new technologies and then applying them to the current product mostly unsure how it would work but that it was somehow cool. This was a Frankenstein's monster of software. It had 2 pass pre-build (making it insanely difficult to debug), 2 view rendering engines, 3 client presentation layer libraries, 2 database interface libraries neither worked well enough so there was a lot of SQL statements as well, database schema so poorly designed that an average query required 2 inner joins and 2 outer joins (many were so poor that response times for a web page under 1 user load would easily go over 30 seconds); it was a mess. Load tests with more than 5 concurrent users often never completed and scaling was pretty much buying lots of hardware and hoping it was enough. When asked why not stick with one technology for each area, he said that he was looking for the right one but hasn't had time to go back and unify them. He saw no problems with 20-30 second reply times and claimed it just needed some tweaking. And of course he did Quoth The Donald... nevermore.

What The World Needs Now

There are many more cases but you get the idea. Premature optimization is not writing efficient code up front (that is desirable), it is writing code that is specifically optimized without knowing the big picture. Premature optimization is not an excuse to write poor code or build on a poor design. It means: Don't spend too much time on minor optimizations which yield negligible results.

Efficient well written code at early stages of development can be a difference between a successful project and a long, protracted code cleanup.

Poor Donald... we hardly knew you.

Running the code through PVS-Studio

2011-01-24T10:15:00.000-08:00

I was generously offered a temporary license to PVS-Studio to do static analysis on my code and the following is the result.

The Hardware

OS: Windows XP Professional x64 Edition (5.2, Build 3790) Service Pack 2
Processor: AMD64 (8 CPUs), ~2.7GHz
Memory: 4094MB RAM

Performance was not much of an issue and analysis took a few minutes per project, but I figured I should port the hardware used for reference.

The Install

Install was painless and it integrated itself into Visual Studio 2010 as a menu item. You can analyze a file, a project or whole solution. You can also load/save analysis.

The Analysis

I analyzed most of the base libraries used by AObjectServer, then base modules and then the actual server code. I also analyzed the load test tool I use (which comes as a command like tool and a Win32 GUI). Overall it was 11 DLLs, 3 EXE. Analysis per project is about 2-5 minutes (depending on files I suppose).

The Results

Results are divided into 3 levels.

Start with Level 1 issues.

MESSAGE: V550 An odd precise comparison: - 1.0 == m_stateTimeLimit. It's probably better to use a comparison with defined precision: fabs(A - B) '<' Epsilon.
CODE:
#define INVALID_TIME_INTERVAL -0.1
return (INVALID_TIME_INTERVAL == m_stateTimeLimit || m_stateTimer.getInterval() < m_stateTimeLimit ? false : true);

FIX: return (0.0 > m_stateTimeLimit || m_stateTimer.getInterval() < m_stateTimeLimit ? false : true);
NOTE: This is definitely a good find

MESSAGE: V550 An odd precise comparison: 0.0 == difftime (t). It's probably better to use a comparison with defined precision: fabs(A - B) '<' Epsilon.
CODE: return (0.0 == difftime(t) ? true : false);
FIX: According to the API reference return of 0 from difftime
NOTE: This is a tougher one, this tests to see if difftime returns 0 which means that it is the same time. I did notice that this function was operating on time_t structure which defined on Windows as __int64 so comparing to another time_t should be fine. I made this change, but I am not sure if other operating systems may use something else for time_t.

-----------

Next are Level 2 issues. Almost all were V112 type that flags usage of numbers that relate to memory sizes (like 4, 8 32, etc); all of these were false positives for me. Few of the messages were interesting.

MESSAGE: V401 The structure's size can be decreased via changing the fields' order. The size can be reduced from 32 to 24 bytes.
CODE: in dbghelp.h (part of Microsft SDK)
typedef struct _MINIDUMP_EXCEPTION_INFORMATION64 {
DWORD ThreadId;
ULONG64 ExceptionRecord;
ULONG64 ContextRecord;
BOOL ClientPointers;
} MINIDUMP_EXCEPTION_INFORMATION64, *PMINIDUMP_EXCEPTION_INFORMATION64;
FIX: None, it's part of the Microsoft SDK but interesting.

MESSAGE: 2776 V801 Decreased performance. It is better to redefine the first function argument as a reference. Consider replacing 'const .. path' with 'const .. &path'.
FIX: I accidentally used pass object instead of pass by reference, while not critical in this instance since it was part of initialization call, this message is very useful.

MESSAGE: 297 V112 Dangerous magic number 4 used
NOTE: The tool is fixated on numbers that may be associated with memory word sizes like 4, in all my cases 4 represents a 4 and not a memory size (I use sizeof if I need word sizes). So this resulted in many false positived and probably should have been level 3.

-----------

Now on to Level 3 here are the more interesting ones.

MESSAGE: V547 Expression 'findFromFront (p) >= 0' is always true. Unsigned type value is always >= 0.
CODE: AASSERT(this, findFromFront(p) >= 0);
FIX: This was not needed since size_t is the result from findFromFront and it is always >=0, this I suspect is leftover code from changeover from int to size_t.
NOTE: Good to clean dead code even if it is harmless

MESSAGE: V547 Expression 'moveSize < 0' is always false. Unsigned type value is never < 0.
NOTE: These were very useful since they are a result of another porting project that replaced int with size_t and went from signed to unsigned. Most of the warning were harmless and inside assert statements but nevertheless, cleaner code is always better.

MESSAGE: V509 The 'throw' operator inside the destructor should be placed within the try..catch block. Raising exception inside the destructor is illegal.
NOTE: Great catch, this was a result of a conversion from assert() to a macro which throws an execption and this was inside a destructor which is a bid thing.

The Summary

Overall I liked the tool, it did find a few non-critical issues with the server code, but to be fair I have ran most of my code through FlexeLint and BoundChecker (until my license expired that is). I also have Visual Studio warning level 4 turned on for debug builds and that catches a lot of issues.

The main benefit of PVS-Studio is that it is good at finding issues that may affect porting 32-bit code to 64-bits.

Faster, lighter and elsewhere

2010-11-18T07:29:00.000-08:00

I have been slowly making progress on AOS, mostly in performance and stability area, so not much to update. I run AOS as my personal web/app server and it has been putting up amazing uptimes. It starts with the OS, leaks no memory and restarts with OS, no incidents of watchdog restarts at all. This is a good thing, given that a large part of the requests are mangled headers, zombie scans, search engine robot scan and other non-website related traffic.

While the product is very niche to websites that require the fastest execution of code or native integration with existing C/C++ code, it still surprises me when I get a comment/feedback email from a place where I did not think this would be used.

The only thing I have changed in the last few months was to upgrade the solution/project files to Microsoft Visual Studio 2010, those will be part of the next release or if you need them sooner, email me and I'll put them up.

Eclipse and Perforce do not mix

2009-05-24T06:09:00.000-07:00

Spent last 2 weeks fighting with the perforce eclipse plug-in (war is not over, I even submitted an enhancement request to support .p4ignore). I wish I knew how poor perforce works with eclipse, I would have fought again it. As a source control it works fine as long as you use their GUI tool, but subversion it is not.

I have used a lot of source control software in my life: CVS, subversion, ClearCase, SourceSafe, Mercurial, Git, perforce and few others for short time.

My favorite has to be subversion. In 7 years of using subversion for my personal development, I have never had any problems. It works from command line, with windows explorer, with eclipse, with visual studio... it just works as you expect it, no fuss, no config woes. Just point it to a URL and you are done.

I want to preface this by saying that eclipse is an excellent IDE, the eclipse plug-in which is not associated with eclipse at all, and is written by perforce team.

Some things with perforce that I find unacceptable for commercial software:
- No ignore file (this is the first source control software that I found that doesn't have ignore), there are times you need to mark a local file so that it is not added to source control (mostly config files that have been customized for the local environment). Eclipse plug-in allows you to create .p4ignore but their GUI tool does not support it.
- Eclipse integration is shameful. There seems to be non-transactional nature to the plug-in, so when a problem is encountered you never know what state the file is in and even worse some problems fail silently. Login issue is the main cause, after the login expires eclipse plug-in starts silently failing unless you restart it; so now I close my eclipse before I go home and open it again when I get into work (annoying).
- Weird states created by eclipse plug-in, if the file is in read-write state due to an error, you are now in an odd state when you can't merge with latest, can't checkout and can only revert to the latest (while backing up your file) then using BeyondCompare (which is an awesome tool) to merge your changes in.
- Use of RO flag for management of check-ins, it feels like its 1995 and SourceSafe again, this flag can get clobbered by some editors or eclipse plug-in when checkout fails.

UPDATE: It has been a year and Perforce managed to fix many of the quirks and works a lot better with Eclipse now, in November of 2010.

Version 1.7.0.0 ready for prime time!

2009-03-11T10:46:00.000-07:00

Lots of evening and weekends spent on getting more stuff in. Most of the changes are internal but I did add a nice visual admin site viewer using jQuery plugins.

After battling windows service code I finally got it working well enough that I feel comfortable using it instead of the console mode. It did take a while to figure out all the little kinks that arise when writing services and having to deal with Microsoft's security system (it's like trying to play Spelunker, read the Gameplay section).

I also decided to organize the entire website into the _website folder which makes getting up and running trivial (not that it was hard before, just now everything is under one folder and you can switch website roots easily which not having to copy config or resources). Something that people that run multiple instances have requested.

Onward and forward...

2008-12-08T16:43:00.000-08:00

Progress

So I added a simple memory tracking class to the server executable (debug mode only and 32-bit at the time, it uses asm directive which is not supported in 64-bit mode so I have to come up with creative ways of accessing the 64-bit registers for the stack traces). Anyhow, it wound up being extremely useful in reporting memory leaks (since I am still working on saving up enough to buy BoundsChecker).

With OpenSSL

First one I found was in how I was cleaning up the SSL library (while most are global lifetime allocation and not really a big issue, they make the reports ugly with 600+ unallocated objects). I found various API calls to explicitly free up memory on shutdown and brought it down to only 2 leaks:

From ASocketLibrary_SSL.cpp:

ASocketLibrary_SSL::ASocketLibrary_SSL()
{
SSL_library_init();
ERR_load_crypto_strings();
SSL_load_error_strings();
}

ASocketLibrary_SSL::~ASocketLibrary_SSL()
{
ERR_remove_state(0);
ENGINE_cleanup();
CONF_modules_unload(1);
ERR_free_strings();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
}

With GdLib

This one was actually quite serious in applications that generate dynamic images. While this is a good example where C++ destructor would have been perfect, the gdlib API is in C and thus you have o find appropriate functions to call:

From AGdCanvas.cpp:

AGdCanvas::AGdCanvas(int sx, int sy)
{
m_GdImagePtr = gdImageCreateTrueColor(sx, sy);
}

AGdCanvas::~AGdCanvas()
{
gdImageDestroy(m_GdImagePtr);
}

I was using gdFree() which unfortunately did not release the sub-objects and thus leaks a bit of memory.

Current State

At this point there are no known leaks and I ran the server under a moderate 20 client load for an hour with no variance in memory used (~30MB in full debug build, release build memory footprint is ~14MB). I disabled all caching for the test to make sure that everything was being allocated and deallocated correctly.

Miscellaneous Notes

Next version looks like it may be done this month, if I can find enough weekend time to do more testing and validation. I may rewrite the threaded queue that handles persistent sockets that are pending data of the next request, currently it uses the synchronization model from std::list and I replaced it with my home-grown ABasePtrQueue which is faster and has built-in ability to synchronize modification if given an ASynchronization object when created.

Finally

I've been spending time reading my new Lua Reference book and planning some more sample code with it. I really like Lua as a scripting language and at this point its speed and ease of embedding is tops in my list.

Overall, I was able to get majority of the sample calls to execute in AMD for that quad-core processor and dual-core processor and motherboard; made client and server with those parts and they prodly wear the "Powered by AMD" stickers!).

AOS Rhino 1.6.0.0 is coming soon

2008-11-26T08:02:00.001-08:00

I have been busy extending the I18N and L10N abilities and reorganizing the web content locations to make more sense (along with a slew of bug fixes and enhancements).

Now I just have to do load testing and flush out any anomalies and release it soon. This should be a very stable release and very usable with multi-language content.

Stay tuned...

The big and silent release

2008-09-26T13:12:00.000-07:00

On September 24th 2008, the first official release of AOS Rhino 1.5.0.1 hit SourceForge.net and is available for download. I still want to do heavier load testing and write more samples, but might as well throw it out into the wild and see what happens.

I have been load testing, debugging, cleaning up and documenting for a few months and it's as good and complete as it can be for the first official release.

There is still a lot that I want to do, but all that is additions going forward and I promise to not change the base API for any 1.5.x.x version, 1.6.x.x is fair game!

64 bit in a day!

2008-06-04T15:14:00.000-07:00

I finally sat down and decided to port my code to 64 bit. I created a new configuration for x64, added Win64 to the predefines and fired off the build. And then it dawned on me, I have 3rd party libraries linking in a few projects (python, lua, gdlib, zlib, xerces/xalan, mysql, sqlite3, odbc and openssl). So I had to make quick decisions of whether to continue or stay with 32bit for a bit longer.

lua: using the actual source to build a DLL, no problem.
sqlite3: using the actual source to build a DLL, no problem.
zlib: 64 bit library I found and linked, no problem.
mysql: had to install the 64 bit version to get include/lib, no problem.
odbc: already native, just had to make a few variable adjustments in the code.
openssl: took a bit of wrestling with the code (assembly file had a syntax error and makefile was including an incorrect static library), in the end I build the 64 bit libs and all worked out.
gdlib: this was a bit tricky, since I could not find 64 bit libs, I had to build it myself inside my code (as I did with lua and sqlite3) but it then needed libpng and libjpeg, which had to be included. After some more wrestling with the config I managed to build it successfully.

And now for the casualties:

python was on the chopping block for a while, I really like lua as an embedded language and python for scripting in the shell. So I made a decision to not have python support from within the app server.
xalan/xerces has been a bit problematic from the beginning and multi-threaded tests occasionally failed which never happened with MSXML. I could not find 64 bit libs and the build system was depended on Configure, so I decided to not include it and rely on MSXML which is way faster and more stable on Windows. When I port to linux I will probably go with libxml2 and libxsl.

After all that the server itself built without a hitch and I was able to start and test it. I surprised myself that there were no portability issues with 64bits compile.

Made me so confident that I installed VMWare player with Ubuntu on it and created a makefile that built 1 of my source files... with about 300 to go; hey it's a start. I suspect the toughest part is going to be porting the synchronization, socket and file based classes; I have been on projects where I had to port code from windows to unix, so it shouldn't be too tough. The 3rd party stuff i have so far is pretty portable, so it should not present a big problem.

My biggest dilemma is what do I call my app server?!
Do I need an animal/plant/insect/object mascot?

All the good names that represent fast and lightweight have been taken by products that are often neither...

2008-06-04T14:55:00.000-07:00

I have been busy for the last few months profiling, linting and optimizing the code. After numerous runs I have convinced myself and (used lint to prove) that there were no memory leaks in the code.

But I was still not convinced, so a friend suggested I run it through his DevPartner BoundsChecker to verify. After a bit of mucking around and restoring old MSDev 2005 vcproj files (apparently BoundsChecker doesn't work with MSDev 2008 at the moment or at least his version didn't). We ran it through a kitchen-sink style test and asked server to exit (I made sure you can do that via admin site). It did so and no memory leaks reported, and they all rejoiced and ate the minstrels. I tend to be very picky with code and was pretty sure that there were no leaks but had to be sure, it's a server afterall and a leak would adversely affect uptime.

Speaking of uptime, the server has been up for over a month (it gets bounced not because there are problems but rather I keep rebuilding it with newer features). I do get a decent number of hits mostly from zombies and bots looking for the latest PHP exploit (I am not kidding, 10-30 attacks on PHP per day, everyday, on my server which obviously doesn't even run PHP). It also helps that many of the HTTP requests are malformed or very long (like that Exchange bufferoverflow one I get every so often) and it tests lots of error handling. And all those port scans. Just more fodder for testing.

Lua Lua, Everywhere!

2007-10-16T18:35:00.000-07:00

Lua is now embedded in AOS, it was a lot easier than I thought. The code is ANSI C so I embedded it into a DLL with ability to dynamically load Lua libraries. I added it to the templates which now behave similar to JSP pages with Java embedded in them. For example:

header.html

<>Page for %[LUA]{{{
print(
aos.emit("/root/REQUEST_HEADER/ACookies/username")
);
}}}[LUA]%< /sometag >

Since you can combine a set of templates to make a full page, you can have a common header/footer/etc pages that are shared by many other pages. And here aos.emit() extracts from the XML model the cookie named username and
using alibrary.print() function writes it to the output stream.

It's very easy once you get used to Lua (and in spirit of C almost everything is defined via libraries); so getting used to it is actually very easy. Try it out: http://www.lua.org/ and if you are not completely satisfied, I'll refund your ... umm... it's free!

The benefits of PHP and testing

2007-07-23T14:08:00.000-07:00

I have never liked PHP as a language due to the extremely high number of exploits floating around on the web, but recently I found it to be quite useful as a QA tool. My application server is constantly getting hit with bots that are looking for one PHP exploit or another. They range from core PHP attacks on admin functionality, help system, proxy functionality and other random PHP scripts that ship with various versions (most trying to run some shell command). On top of that there are a lot of PHP 3rd party programs (forums, blogs, etc) that either open more holes or re-expose existing ones. I worry about any site running PHP, they are exposing themselves to a lot of support and maintenance hassles.

However, this is good for me. I get so many hits from bots (and sometimes script kiddies) looking for various holes in PHP that the requests are often malformed, formed incorrectly, contain invalid data, etc; not to mention the constant port scans from not so nice scanners that don't always follow TCP/IP standards (but that's all good, more fodder). This constant assault provides ways I can fix unexpected and often incorrect behavior on my server that results from inexperienced web hackers or automated bots.

To be fair, I do get a rare MS exchange hole where the Authorize header is extra long (for IIS) and tries to exploit the mail server hole. I have written my HTTP parser to be quite heavy duty so it's just a blob of data and if it is too long it will be discarded looking for next CRLF (instead of trying to read it into a fixed buffer which is how a lot of buffer overflow attacks are done).

Long live PHP in all of its poorly coded glory.

And then there was 1.0

2007-03-04T04:05:00.000-08:00

I have finally met the 1.0 milestone (albeit still beta).

The biggest change are:
- added HTTP 1.1 pipelining (IE7 still doesn't support it, but the browsers that do benefit from much better performance)
- upgraded to the latest sqlite3 (with source), gdlib and mysql includes
- sped up he execution by converting the 3 separate execution queues in 1, which is not as complex as I thought it would be and is way faster (getting single digit millisecond execution times on semi-complex calls on an ancient pre-millenium computer).
- ran lint on the code and cleaned up a bunch of stuff
- added event visitor and improved administration, with this I can track any request through the execution path to determine if there were any problems (I like it a lot, but may add a way to turn it on/off to speed things up more, performance at this point is not an issue, but more is always better)

Things to do:
- finish gzip code (compression only works on mozilla, IE is behaving strangely)
- write more sample applications (need to finish the blog and forum examples)
- externalize DB queries to be database specific in a resource file (it is very easy to switch databases, but the SQL format varies ever so slightly)
- add asymmetric static content cache in case the server is used as HTTP/HTTPS server (which is does very well but caching would make it even faster).
- add ability to view server logs via administration interface (not urgent since you can always just open the file yourself, but a convenient feature)

Overall the server is stable and very fast, which is the goal of this project. Once I have more functionality added I will start thinking about how to release it.

Execution engine is finally done

2006-10-30T11:33:00.000-08:00

Took me long enough, but with a new job, time has been scarce. Everything is running and really fast (simple templated page takes 20ms to generate using debug code and <10ms in release). The only thing I need to tune is the number of queues and threads per queue to optimize performance.

Next step is to create modules (similar to the way context jars work in tomcat), currently the content is deployed by a script, but being able to package it in a zip and let the execution serve deploy (again like tomcat) is a nice feature.

Using SQLite3 as a database has been very handly, it is a single file database that I can just drop wherever I need without having to install SQL server and deploy data. For larger scale I would look to MySQL or postgreSQL, but for development it's a joy to use.

I am also going to switch away from CVS, getting tired of it. I may switch to SVN or something else that will allow me to use a networked drive as a repository and allow me to move files without having to much with the file system. CVS is great but lacking, I have used ClearCase in the past and really liked it, it does cost a lot.

Indexing and query string

2006-08-04T08:27:00.000-07:00

Haven't had much time to do any further development due to a new job, but I'll find a few minutes on the weekend to finally finish the queue execution system. The server is running well with just one queue per stage with N threads per queue. I broke down and started using spin locks as the performance under load is superior to a critical section (which is even faster than a mutex obviously).

The big change will be the removal of application specific objects and relying more on the AOSContext to manage data. This change will allow executions to be even more parallel which is essential for a very high request volume.

The other fun thing I have been working on is building an indexed blob for query strings that will allow database query against it without having to parse the name/value pairs. I have been looking into a simple database index structure to build it, hopefully once finished it will allow fast lookup of name/value pairs in a database without extensive logic.

Caching time

2006-07-03T10:17:00.000-07:00

Started re-writing the cache code for AOS. About 8 years ago I read an article about using an asymmetric cache that is optimized for storing lots of little objects and as the size went up the probability that the object is cached went down. Performance tests have shown that this is an ideal cache for the web as majority of the objects are <100k,>1M is not worth caching (unless it is expected to receive a large number of requests).

I have held off rewriting the cache until I finished writing an efficient hash-map, for now it's a simple hash of rb-trees and a relatively quick hash function. Insertion into the hash map is still synchronized, but it only needs to lock one of the maps and with a decent amount of maps (usually a prime # to minimize hash collisions, like 13 or 17 for small-medium site or 29 or 31 for a medium to large site) the locking is not a problem and if locking is done at the exact time of the insert it is rather quick.

I have tried using spin locks for this but the insertion load is high only on startup and in steady state spin locks are not as efficient (CPU cycles needed to execute code), so it may be possible to swap from a spin lock to mutex after some time, but it's probably adding too many moving parts. Inserting definitely needs a lock since the internal structure of the rb-tree may change due to a re-balance and cleanup thread may need to lock it during physical removal of the nodes, but during reads we can avoid any locking. This raises an interesting race condition of reading something that cleanup thread just removed. For this I am using a temporary bucket that gets physical nodes that are pending removal at the first cleanup iteration and removes from the mmap, this allows the node to exist for a while and for references to drop off if any (the cleanup thread would only remove seldom used items and if the items is in use it is not likely that it is going to be removed in the first place, but there is always a tiny chance). This staged removal should eliminate any chance of a race and it is even possible to iterate this bucket and put back any items that were actually accessed during the isolation time, but again it is such a small chance it may not be worth doing.

Hopefully I can get this cache inserted into AOS by end of summer, it should make static content reads that much faster. Currently a simple HTTP GET request is taking <1ms and I can't effectively measure it using OS tick counts, I will switch to a higher rez timer to see how long it really takes (and all this on a 6 year old P2-666MHz machine, I can only imagine how well it does on a dual opteron).

constness of it all

2006-06-29T13:06:00.000-07:00

As a designer of libraries and code that others use, I have come to an impasse with Java. I am convinced that it was originally designed for ultimate portability and application development and not for efficiency and library design. One sticking point that has bothered me since I got JDK 1.0 back in 1996 was the absence of const, commonplace in C/C++. const keyword means something is not allowed to be changed (barring any void* hacks).

So if someone writes:

const char *p;

You know that this is a pointer to a char array that cannot be changed. But const is not always this easy to understand, lets look at this example:

const char *const p;

This means that the pointer contents cannot be changed and the pointer value (memory location of the char data) cannot be changed. Further you can really complicate things with:

const char *const get(const char *const) const;

Here a class member function accepts a pointer to a char array that cannot be changed and pointer that cannot be reassigned, it returns the same and on top of that the class data cannot change. Little much but const can be a beautiful thing in the hands of an interface designer.

Let's take a simple example of a class that is a global cache that you cannot change:

class GlobalCache
{
public:
const ObjectData& get(const KeyName&) const;
};

Simple enough you give it a reference to a key object and it looks up and returns a constant ObjectData reference without changing its own contents. The fact that reference is used to pass data around is efficient because no memory is allocated/copied in the process. The method is const (the last const in that declaration) so nothing is written to the object during that call and the most important const of them all is the first one. It means that the data you get you cannot change in any way. Why is it important? You don't need to synchronize access to that object, in a multi-threaded environment you can have as many threads as you want reading that object without a need to synchronize it in any way.

There is no way to do this in java. The closest thing is to return a copy of the ObjectData (and ObjectData can be huge):

class GlobalCache
{
public:
public ObjectData get(const KeyName key) {
return new ObjectData(someinternalObjectaData;
}
};

Java has to make a copy of this object to guarantee consistency. If you return the actual object then anyone can change it which can have very unexpected results if the system is multi-threaded (object consistency is lost). Before you mention final keyword, remember that it means the reference cannot change and makes no guarantees about the object itself.

Not having a const means you need make a lot more complicated of a design to guarantee that the code behaves as intended (because you can never expect the users to do what is expected, they will always do what is not). This is one of the reasons java code tends to bloat in the class count area and it is never fun to have several classes doing what really one class should have done.

I am hoping in one of the upcoming java versions implements a const-like behavior (afterall 5.0 has template-like behavior)... More and more java is starting to look like C++. Funny how that happens.

2006-06-26T10:48:00.000-07:00

AOS (AsynchObjectServer) is undergoing tons of changes, I decided to rewrite the processing queues to modularize the server a bit more. The first 2 stages are: HTTP reader and socket selector; once a listener socket accepts a connection (either HTTP or HTTPS, both treated identically having a common base class AFile_Socket_SSL is a child of AFile_Socket). SSL sockets are slightly different, but when data is available we can use the AFile_Socket_SSL::read and the cyphers are handled automatically, so the socket selector queue just needs to determine that there is data waiting and the HTTP reader queue will then process the socket. Once HTTP reader determines the HTTP request header is complete, it creates a AOSContext object based on AOSRequest and pushes it into the first "application logic" stage.

The AOSInputProcessor-based queue which then handles the input; primarily it splits the query string, appends FORM data if needed or handle a custom request document.

Once finished it pushes the context into AOSMOduleExecutor-based queue which executes applicable modules for the given request command and once that is finished it will push context to the next queue.

AOSOutputGenerator-based queue then performs the output generator, by default the XML document is returns and nothing is needed to generate but it can do XSLT or template substitution/concatination or whatever output is needed.

Splitting AOSRequest (socket request) and AOSContext (application context) queues allows the server to have multiple entry points into application execution, since AOSContext relies on AFile interface it allows anything that behaves like a file to work as a recepient of the execution logic. AFile interface is implemented for string buffer, physical file, and stdc++ IO stream.

This is a rough overview of the queue chains (I omitted the error queue). Each queue is really a set of queues with N threads working them asynchronously. Overall throughput of this setup (which is very much like SEDA but I didn't know about their papers until I read them few weeks ago) is quite high and in my initial load tests was about 3-4 times faster than Apache mina-0.9.4. I am going to rewrite memory allocation optimizations and improve concurrency then retest. I am hoping to have a working server again in a few weeks.

Initium

2006-06-26T10:26:00.000-07:00

So this is a continuation of the progress report on Asynchronous Object Server I have been writing for the last 9 months or so. Rest of the progress notes are at http://www.achacha.org/aos/, blog format is probably easier to maintain and read.