Showing posts with label 16-bit windows. Show all posts
Showing posts with label 16-bit windows. Show all posts

Sunday, 18 October 2020

Fast Hardware Hides Many Sins

Way back at the beginning of my professional programming career I worked for a small software house that wrote graphics software. Although it had a desktop publisher and line-art based graphics package in its suite it didn’t have a bitmap editor and so they decided to outsource that to another local company.

A Different User Base

The company they chose to outsource to had a very high-end bitmap editing product and so the deal – to produce a cut-down version – suited both parties. In principle they would take their high-end product, strip out the features aimed at the more sophisticated market (professional photographers) and throw in a few others that the lower end of the market would find beneficial instead. For example their current product only supported 24-bit video cards, which were pretty unusual in the early to mid ‘90s due to their high price, and so supporting 8-bit palleted images was new to them. Due to the large images their high-end product could handle using its own virtual memory system they also demanded a large, fast hard disk too.

Even though I was only a year or two into my career at that point I was asked to look after the project and so I would get the first drop of each version as they delivered it so that I could evaluate their progress and also keep an eye on quality. The very first drop I got contained various issues that in retrospect did not bode well for the project, which ultimately fell through, although that was not until much later. (Naturally I didn’t have the experience I have now that would probably cause me to pull the alarm chord much sooner.)

Hard Disk Disco

One of the features that they partially supported but we wanted to make a little more prominent was the ability to see what the RGB value of the pixel under the cursor was – often referred to now as a colour dropper or eye dropper. When I first used the feature on my 486DX PC I noticed that it was a somewhat laggy; this surprised me as I had implemented algorithms like Floyd-Steinberg dithering so knew a fair bit about image manipulation and what algorithms were expensive and this definitely wasn’t one! As an aside I had also noticed that the hard disk light on my PC was pretty busy too which made no sense but was probably worth mentioning to them as an aside.

After feeding back to them about this and various other things I’d noticed they made some suggestions that their virtual memory system was probably overly aggressive as the product was designed for more beefier hardware. That kind of made sense and I waited for the next drop.

On the next drop they had apparently made various changes to their virtual memory system which helped it cope much better with smaller images so they didn’t page unnecessarily but I still found the feature laggy, and as I played with it some more I noticed that the hard disk light was definitely flashing lots when I moved the mouse although it didn’t stop flashing entirely when I stopped moving it. For our QA department who only had somewhat smaller 386SX machines it was almost even more noticeable.

DBWIN – Airing Dirty Laundry

At our company all the developers ran the debug version of Windows 3.1. enhanced mode with a second mono monitor to display messages from the Windows APIs to point out bugs in our software, but it was also very interesting to see what errors other software generated too [1]. You probably won’t be surprised to discover that the bitmap editor generated a lot of warnings. For example Windows complained about the amount of extra (custom) data it was storing against a window handle (hundreds of bytes) which I later discovered was caused by them constantly copying image attribute data back-and-forth as individual values instead of allocating a single struct with the data and copying that single pointer around.

Unearthing The Truth

Anyway, back to the performance problem. Part of the deal enabled our company to gain access to the bitmap editor source code which they gave to us earlier than originally planned so that I could help them by debugging some of their gnarlier crashes [2]. Naturally the first issue I looked into was the colour dropper and I quickly discovered the root cause of the dreadful performance – they were reading the application’s .ini file every time [3] the mouse moved! They also had a timer which simulated a WM_MOUSEMOVE message for other reasons which was why it still flashed the hard disk light even when the mouse wasn’t actually moving.

When I spoke to them about it they explained that once upon a time they ran into a Targa video card where the driver returned the RGB values as BGR when calling GetPixel(). Hence what they were doing was checking the .ini file to see if there was an application setting there to tell them to swap the GetPixel() result. Naturally I asked them why they didn’t just read this setting once at application start-up and cache the value given that the user can’t swap the video card whilst the machine (let alone the application) was running. Their response was simply a shrug, which wasn’t surprising by that time as it was becoming ever more apparent that the quality of the code was making it hard to implement the features we wanted and our QA team was turning up other issues which the mostly one-man team was never going to cope with in a reasonable time frame.

Epilogue

I don’t think it’s hard to see how this feature ended up this way. It wasn’t a prominent part of their high-end product and given the kit their users ran on and the kind of images they were dealing with it probably never even registered with all the other swapping going on. While I’d like to think it was just an oversight and one should never optimise until they have measured and prioritised there were too many other signs in the codebase that suggested they were relying heavily on the hardware to compensate for poor design choices. The other is that with pretty much only one full-time developer [5] the pressure was surely on to focus on new features first and quality was further down the list.

The project was eventually canned and with the company I was working for struggling too due to the huge growth of Microsoft Publisher and CorelDraw I only just missed the chop myself. Sadly neither company is around today despite quality playing a major part in the company I worked for and it being significantly better than many of the competing products.

 

[1]  One of the first pieces of open source software I ever published (on CiX) was a Mono Display Adapter Library.

[2] One involved taking Windows “out at the knees” – not even CodeView or BoundsChecker would trap it – the machine would just restart. Using SoftICE I eventually found the cause – calling EndDialog() instead of DestroyWindow() to close a modeless dialog.

[3] Although Windows cached the contents of the .ini file it still needed to stat() the file on every read access to see if it had changed and disk caching wasn’t exactly stellar back then [4].

[4] See this tweet of mine about how I used to grep my hard disk under Windows 3.1 :o).

[5] I ended up moonlighting for them in my spare time by writing them a scanner driver for one of their clients while they concentrated on getting the cut-down bitmap editor done for my company.

Monday, 19 October 2009

DDE Is Still Alive & Kicking

Dynamic Data Exchange (DDE) is an ancient inter-process communication (IPC) mechanism carried over from the 16-bit Windows days. Post millennium Windows programmers are probably more used to a diet of COM, but there still seems to be life in the old dog yet. Maybe there are less people around to answer questions on DDE, or perhaps the other old timers are ignoring them in the hope they'll go away because I seem to have had more emails on the subject this year than ever.

I presume the reason the questions are still coming my way is because I have a number of freeware tools on my website aimed at working with DDE Servers that are still actively supported:-

  • DDE Query is my oldest tool and is a GUI based utility for sending requests and creating advise loops on items. It was written as the main test harness for my DDE library.
  • In contrast DDE Command is my most recent addition and is the console based counterpart to DDE Query. It also provides the ability to invoke XTYP_EXECUTE transactions.
  • In between the two is my DDE COM Client - an automation compatible inproc COM component that can be used as a DDE Client for scripting scenarios, such as VBScript.
  • The final, and second oldest utility I provide, is a Network Bridge to allow DDE Servers to be accessed remotely. It is entirely transparent to the client and server, unlike the built-in Windows NetDDE service.

These all use my own C++ DDE library which in turn is based on the DDEML C-API library that Microsoft provides. Under the covers DDE is just a message based protocol that uses Windows messages to encapsulate a connection between two applications – known as a Conversation. Data is passed by allocating it with the old GlobalAlloc() API, and the format is determined by either using a standard clipboard format, such as CF_TEXT or a custom one via RegisterClipboardFormat(). The fact that it is message based is also its biggest limitation as it means you cannot share data between machines – you can't even share data across desktops on the same machine. The only book I came across on the subject was Ole 2.0 and Dde Distilled by Al Williams, but it covers the topic pretty thoroughly.

Before working in finance the only time I had come across DDE was when writing an installer back the mid 90's. Under Windows 3.x you used DDE to communicate with Program Manager (the shell that today is known as Explorer) so that you could create a "Program Group" and the icons for your application. You can still see this legacy today if you fire up DDE Query and use the "Server | Connect…" option, where you will spot a server called PROGMAN with a single topic also called PROGMAN. If you open this conversation and request the item "Accessories" you will be shown a CSV formatted text block with the programs from the Accessories Start Menu folder.

Once I started working in finance I discovered that Excel was the traders tool of choice. Excel can pull data from a number of sources, but the legacy option for real-time data was DDE. The big providers like Reuters, Telerate & Bloomberg all provided tools to allow you to feed their financial data into a spreadsheet and the in-house software we developed followed the same architecture. Although COM was probably Microsoft's promoted technology, DDE felt far simpler to implement.

These days I don't work with that kind of real-time feed, but the questions I do get on DDE always seem to have RIC's in the examples, so I guess that it's the financial industry that is keeping this prehistoric mechanism alive.