Vendors: a sad and unresolved tale

I am involved in a project to put one of our existing services on the web. Two coding teams are involved–one vendor wrote the in-house app (I’ll call it the “back end” below, though actually it’s a full client/server system), while the other organization is responsible for the web interface. For most purposes, the web piece is just another interface on the c/s system; the delivery method differs, but the database, workflow, and document creation operations are shared. If the work is processed by our staff (through a GUI), the service will be delivered on paper; if it is processed via the web, it will be delivered as an Adobe Portable Document Format file.

Some History

Both applications are new. The in-house application replaced an obsolete system. (It was obsolete in three senses–by a change in the law, by aging technology, and by its use of an obscure coding environment.) Originally, the web piece was to be sub-contracted by the vendor writing the server code, but (our) organizational politics resulted in two parallel projects, with four project managers (a business manager with one vendor PM reporting to him; the other vendor PM reported to an IT manager). This did not work out well, even though everyone involved was apparently competent and the vendors were large companies you’d certainly recognize. The in-house piece went to production pretty much on schedule; the web piece is now nearly a year behind its intended launch. This is a little frustrating. High management has been patient, but we all feel pressured.

Further organizational politics intervened. In January, the web vendor was jettisoned and replaced by web coders in our organization’s employ. We’ll pass over the ins and outs of this change, but must note that it was several months before the new (IT) web coders properly understood the technical issues in the project. One issue, in particular, has stymied the Go-Live effort: The web piece is able to process the work, but the delivery mechanism was inadequately specified. The specific problem is that the application is capable of producing very large files; while most PDFs created for delivery will be fairly small, we get requests which would create a half-gigabyte file. Delivering those could be a bit of a problem.

The Issue at Hand

Actually, it was worse than that. During their efforts to develop a satisfactory delivery solution, the IT web coders concluded that the architecture of the original design was seriously unstable. We’ve spent the past month or so working that out:

  • The IT coders suspected the problem, and reported it to us.
  • The IT coders tested and verified their suspicions, working with representatives of the original vendor.
  • The IT coders proposed a solution, talked it over with us, and discussed it with the back-end vendor. Only minor changes would be required in the back-end code to implement this proposal.
  • The IT coders quickly determined that there was a performance problem with their original plan, and offered an alternative. This, too, was discussed with the back-end vendor, as it was pretty clear they’d need to make significant modifications to their code for this solution to work.
  • The IT coders began working on the fix.
  • The other day, they announced their code was nearly ready for testing.

Communications Breakdown

Here’s where the story gets fun. My boss (we’ll call him Mel) and I were certain that the back-end vendor had not agreed to their part of the fix, and that they’d not written the necessary code. This was a gut-feeling thing; we just hadn’t heard things we expected about the coding changes. Everyone else was surprised that we thought there was a problem. When we set up a teleconference to directly ask the question, though, it turned out we were right. There was a communications breakdown; even though staff from the back-end vendor had participated in earlier discussions, they’d not fully understood the implications of the discussion. Not a pretty sight. These things happen, unfortunately.

We spent the call discussing alternatives, and asked the vendor to make a recommendation about a best-fit fix. That showed up in Friday’s e-mail; we’ll be discussing it on Monday….