RPG and Programming

Well, I suppose a new chapter in my career as an RPG programmer has begun. My contract ended at the end of January, so now I am looking for a new RPG position, which is not easy when you live near the center of the 48 contiguous states. Being the age I am, I have zero interest in moving to another part of the country that will keep me employed until I want to hang it up in a very few years. So, I have restrictions on my job search. It’s a little easier than the last time I lost a job, because I and my wife are collecting Social Security, but I do need some extra income. I would rather be a greeter at the local Walmart than relocate.

With that in view, I am looking to telecommute. And you may or may not be surprised to know that the market for telecommuting RPG programmers is rather sparse. I have found a few and have had a couple of phone interviews, but as yet it hasn’t gone any farther than that.

What is interesting is the difficulty I have in specifying my desires to the various websites that offer access to people who are hiring programmers. Some are good, others not so much. If you specify “telecommute” or “remote”, the best of them will make a good faith effort to find positions; but even they must have an algorithm that search job descriptions for the keywords specified and ignore context; the text may say “NOT remote” or “NOT a telecommute position” or “local only” or “onsite only”. The site dice.com is at least helpful, when at the very top of the listing it says “Not a telecommute position” in its search for telecommute positions.

Many others are much worse. I did a search and found a position I applied for near Lake Mary, Florida. Now, I must have eight to ten different web sites trying to find me jobs near Lake Mary. When I try to edit the alert specifications, they will not allow me to specify “Anywhere” or “telecommute” as a location or as a keyword. Some of them try to force you to use words like “developer” or “programmer” or “analyst” and will not even allow RPG as a language specification. They want you to use their canned keywords. Apparently, C++, C#, Java, and Python are the only languages being used today. That, my friends, is lousy programming. And keep in mind that this is coming from a diehard “Mainframe” style programmer when I say that, while surfing the Web, I have seen some of the lousiest, bug-filled programs ever created for public consumption. I would have been fired a long time ago if I had written comparable stuff on the platforms I have used.

So my current keyword selection is “RPG remote telecommute”. I dare not use “developer” or “programmer”; if I do, I get a multitude of non-RPG jobs. There is usually no way in the keyword specification (except at dice.com) to specify a keyword as “required”.

I tend to get irritated at some job posts that specify on-site as a requirement, especially when I come from a site where 3 (and soon 4) of the IT staff are remote workers from as far away as Wisconsin and California. That is half or more of the total IT staff. I’m sorry, folks, it simply isn’t necessary anymore. You can tell how diligent and responsive workers are to user needs by their output. You don’t have to see their faces to know whether or not they are doing a good job.

Sometimes the creative juices aren’t flowing very readily. Sometimes it just seems hard to sustain a train of thought long enough to produce a mini-essay. Perhaps I should try what I see some newspaper columnists do – dispense random thoughts, and perhaps a gem will coalesce from all those little bits.

Not long ago, an idea was floated that it would be good if somehow an apprenticeship program could be developed for RPG. Not many schools teach RPG as part of their computer science curriculum. But how do you start something like that? It’s kind of neat when you can teach someone without any programming experience a computer language, as I am indeed trying to do now. But as an industry-wide trend? I don’t see it happening.

I bought one of those $298 Compaq laptops from Walmart a few weeks ago, and I am pleased with it. As it happened, a columnist for ZDNet wrote a column entitled “Culture of cheap: How discount computers cost the consumer.” It provoked a lot of responses, and I was moved to write one – not that it was all that perceptive. The responders were roughly divided into two camps: “You get what you pay for “ and, to quote the American songwriter Ira Gershwin, “It Ain’t Necessarily So.” I was in the second group.

It so happened that somehow, on the Web site, somehow all of the quotation marks I used were converted into question marks. (I love PCs…. NOT) Someone who held an opposite view picked up on this, ignorantly slurred my operating system Kubuntu (’Why don’t you install a real operating system , like Ubuntu’- . Of course, Kubuntu is a variant of Ubuntu created by the creators of Ubuntu), then somehow decided my spelling and grammar were lousy, thus proving that I didn’t know what I was talking about. Since my spelling and grammar were impeccable (not bragging, just stating the facts), I simply had to respond to him with a little enlightenment on Kubuntu and suggested to him that he take my post to a spelling checker and a good English teacher and see if it was really that bad. ( I must be getting either older or smarter; I used to respond to stupidity much more sarcastically.)

Finally, I came across a very neat article about the use of Boolean variables in Basic. Of course Boolean variables are used as a data type in RPG; and indicators are just Boolean variables in disguise. The article noted that in Basic, if you say “PRINT 2=2” it will not give an error, it will print -1, “True” in Basic, while “PRINT 1=2” will print 0, “False” in Basic.

Let me quote his assertion now.

“It seems that relational expressions need not exist only within an IF statement or as a condition of a WHILE loop, yet those are the only constructs in which one ever sees them being used. Which brings us to The Great Unknown Of (Basic) Programming, and you heard it here first:

The relational and logical operators can be used in ANY mathematical context!

A corollary to that is:

Any LOGICAL CONDITION can be expressed as an ALGEBRAIC EQUATION

That means formulas. (It also happens to mean that the keywords IF and THEN are technically redundant, and that any program can be written without them! “

Isn’t that a neat idea! I don’t know for sure how I’ll be able to use it in the future.

So X=2: IF Y<>0 THEN X=3

becomes

x= 2 – (Y<>0).

As he expresses it: “The term (Y<>0) is evaluated. If true, it is assigned a value of -1. The expression becomes X = 2-(-1), or 3. Should Y = 0, then (Y <> 0) would be false, and its value 0. Then, X = 2-(0), or 2.”

Of course since in RPG the true and false values are ‘1′ and ‘0′ and are not really numeric values, it would take a little work to adapt the concept to RPG; and since I’m doing this late at night, my brain is only running at half-speed, and a full elaboration of the concept will have to wait. But to be able to eliminate Ifs and whiles! That way my code could be guaranteed to be obscure! But maybe I’d better go slow on this.

I have read on more than a few occasions that it is important what programming language that you choose to write programs in, because it has a bearing on how you think about programming problems. An RPG programmer will likely approach a report program with an entirely different view than a C programmer. In this case, the RPG programmer would likely be able to take a more direct approach to the program than the C programmer, who would likely need to pay more attention to how to express the precise report layout. On the other hand, while compilers are written in C, I shudder to think of what a compiler written in RPG might look like. And can you imagine an RPG compiler written in RPG? Niklaus Wirth’s Oberon compiler was written in Oberon. Don’t ask me how.

This subject came to mind as I contemplated some of the indicator-laden code written for our system over the past 35 or so years- some original code, some heavily modified and added to over the years.

The code, of course, is hard to follow. It may have been easier to write. It’s easy to write code conditioned on one or two indicators; then a condition or two must be tested for, and all of a sudden you are slowly spun into a swirling maelstrom of competing indicators, repeating your original two as you go down the page and see how many permutations of the other three can be devised, how many of each permutation are used before the combination changes, with an occasional sixth or seventh indicator thrown in for good measure. And of course the output is conditioned on each of those permutations, with individual fields based on further combinations of those and others outside that group.

As I said, some of that code was written in the 70’s, before structured concepts began to be widely used, and certainly before they were applied to RPG. I learned how to do this even before concepts like IF-THEN-ELSE-END were even part of RPG. (See my 5/10/08 post “The Good(Indicators Part 2)”.) Evidently, not too many people read the same magazine articles I did, and even fewer implemented the concepts before IBM began to implement them in RPGIII for the System/38.

What is sad is that even after the concepts were made available, they were not well implemented in our shop. Maintenance of the old code was hard enough, but the more they modified the programs in the style they were written-whether RPGII or RPGIII- the harder they became to follow. And when RPGIV came out in the mid-90s, it was effectively ignored. I think I wrote the first RPGIV programs on our system in 2007.

When you are unwilling to implement new programming ideas, it makes the maintenance task even harder. If, when RPGIV came out, they had begun tweaking the code to make it easier to read, it would be easier to follow now; and sometimes relatively trivial tasks would not be attempted simply because the techniques that made them simple were ignored. By maintaining the old mindset, a lot of time that could have been used progressively was wasted.

A big example of this comes in the area of string handling and editing of numeric data. Of course, the handling of string data on a character-by-character basis has to be done in RPGII, RPGIII, and RPG400 by pushing them into arrays and manipulating them as individual array elements. Formatting of numeric values into, say, dollars and cents, must be done by means of edit codes and edit words, which are character patterns that define the look of a number.

An example: On a line of a subfile of a transaction history, the two numbers to be displayed could be dollar amounts (12345.67) or interest rates (123.4567). In generating the subfile, I found that you could not format the numbers in one of two different ways(using edit words), depending on whether it was to be dollars and cents or a percentage rate. This was unacceptable to the the screen format compiler; you had to pick one or the other, and the dollars and cents version was selected. This, of course, then has to be translated mentally by those who use the screen.

The solution, then, would be to output a value that could be expressed in two different ways. What I did a few years ago, before RPGIV was available, was to write the value to a temporary file (just as I would to a print file), then chain to that file and pull out the newly formatted value for use. To format a date, I would write a date as ‘03/08/09′ to a file, then access that 8-character string. Somewhat of a kludge, but it would work. There are no doubt other ways to do this in RPGII or RPGIII, but they are still likely to be kludges.

But in RPGIV, it becomes a piece of cake: Suppose you have a 7 digit numeric field NBR. You can then define a 10-character field FIELD and two edit words as constants: DOLLARS ‘ , 0. ‘ and PCT ‘ 0. ‘. You then say

IF (dollar value) EVAL FIELD=%editw(NBR:DOLLARS)

ELSE

EVAL FIELD=%EDITW(NBR:PCT)

ENDIF

This FIELD is the value that then goes into the subfile. It was very easy to test, implement, and put into production. The users were very pleased.

Now, this solution that took me only a very short time could have been done a long time ago, but it wasn’t. The possible solutions, based on the way of thinking engendered by the old programming language (RPGII) made a simple correction a daunting task, and evidently it was felt it would take too long to implement.

Now, this was not intended necessarily to be self-congratulating. It was meant to show that not being willing to learn new and better things can have longlasting consequences. More to the point, being too deeply immersed in a certain programming linguistic style can cause one to avoid ideas that could be better both for you and for your users. In an environment like this, it may not be practical to choose between C and RPG; but while choosing new stuff (change) may not always be better, it CAN be.

A few days ago, I was editing a program in WebSphere, the IBM-supplied Windows programming environment. I was scrolling the screen when, without warning, the program died. I was presented with the disappearance of WebSphere, replaced by a message box: “Eclipse” in the Windows header line, followed by the message line “JVM terminated. Exit code=8096.” This was followed by informative details like “Xquickstart” | “Xms40m” |, “Xmx768m”,…    I’m sure you get the picture. Thirty-one lines of error message. To anyone but someone in tech support at WebSphere HQ, totally useless. And if you did a print screen and found an e-mail address to send it to, their first response would be, “What exactly were you doing when it happened?” and the second would be “Could you replicate your error?”; since I wasn’t logging my keystrokes, just doing my job, all I could tell them was “Scrolling my screen” and “No.”

But this is getting to be a bit of a rant. What I really want to mention is how my background affected my reaction.

I started on an IBM System/3, then graduated to a System/34, then System/36, then AS/400. IBM midrange all the way. All of these systems were rock solid. Probably no computer user who has not worked on one of them (except perhaps mainframe people) can appreciate this. In almost 30 years of experience, I have NEVER had a system go belly-up on me. No blue screens of death. No “Illegal operation” errors. If a program crashed, it was because I or some other programmer had made a mistake, and the system usually told me very clearly the source of the error. No “exit code=7096″. If it gave you a message code, it also told you what the code message said.

And even Unix people, users of the other supposedly rock-solid operating system, have no reason to be smug. For example, I remember working with a Unix guy on a project that involved running RPG on a Unix box. In the course of his work, at one point, his cursor disappeared (!)  Not dismayed, he entered: “tty” (and remember, he couldn’t see what he was typing), hit a control key combination, then “sane” (Cute. The keyboard was insane, and he would restore its sanity.) .  And it worked. On the IBM midrange, such shenanigans were unnecessary. You never lost your cursor. And as for Linux, there have been countless times, while I was working in Linux, that the system locked up when I wasn’t doing anything more than what any other normal, non-technical user might be doing.

When you work on operating systems like the Midrange boxes had, you get spoiled. And, I have found, your programming style is affected. It may be the old batch processing mentality getting involved here, but if you have been a long-time user, you tend to be more careful. My Unix friend was of the sort that would throw together code on the fly and immediately run it, without any sort of desk-checking. I suppose he was of the sort that would let the debugger catch the errors, if the program didn’t show that it was wrong by crashing. I and others on my chosen platform tended to desk-check more. I feel that the more you examine your code, the less likely concealed errors will evade your notice. (And the devil-may-care problem solving process seems to be an issue with programmers of both Windows and  Unix background.) Maybe it was better that debugging facilities in early RPG were so lousy- you inserted a temporary DEBUG opcode into your calculations.

I guess when you are on the Midrange, you are less tolerant of casual errors. When your operating system spontaneously self-destructs, you are not overly concerned when your program you wrote self-destructs. Just hit Ctl-Alt-Delete, and everything will be okay.

I know I probably haven’t proven that Midrange programmers (and by extension RPG programmers, since most Midrange programmers write RPG ) are better than programmers on other systems. I probably didn’t prove much of anything. All I know was that my fellow programmers knew exactly what I was talking about when I did a print screen of my Web-Sphere message and gave them a printed copy with my caption: “The difference between the AS/400 and a PC.”

Fetch Overflow is one of those somewhat obscure programming constructs that a “modern” programming language lover would likely scorn or ignore, but it epitomizes what makes a programming language like RPG special.

Fetch Overflow requires Output specifications, which are being used less and less, and it requires the use of an overflow INDICATOR (aaaaaggghhh! See the Java programmer running away into the bushes.) Some RPG programmers may not even know how to use it; more’s the pity.

For those applications that continue to generate reports on real paper, the problem of handling end-of-page conditions, including when to print headings, has always been an issue. RPG’s use of the overflow indicator (which may be OA-OG or OV - OVerflow, get it?) has always been to me a very elegant way to generate proper report appearance.    You can condition lines of print on the indicator, if you use O specs. The system default for printing is a 66 line page, with overflow at line 60; but this can be adjusted to suit the needs of the application. Even with an externally defined print file, you can define a numeric overflow indicator and print lines and do jumps to new pages at page overflow.

There is a column (I believe it is 15 on the standard RPG specification,something different on the RPG IV O spec), where you tell whether it is (D)etail, (H)eading, (T)otal, or (E)xception time output. The next space to the right is where you can put an F, for Fetch Overflow. When this occurs, the programmer is assuming that there will also be some other lines conditioned on the overflow indicator, usually specifications that tell the program to advance to the top of the next page and print defined headings, often several lines of them. If the printing on the page has reached up to or past the overflow line (let’s say line 60), and that line meets the requirements for being printed, the program will print all the lines conditioned on the overflow indicator (let’s say OF), then print the line that fetched the headings, then set off OF.

In the past, often the OF indicator would be used in conjunction with the 1P (first page) indicator. But there is available an even slicker way of handling the headings. Use an *INZSR subroutine, which is automatically executed before any files are read. In that subroutine, insert   SETON   OF   or    EVAL *INOF=*ON.

Now, what happens when the first detail line is read with the Fetch Overflow flag? It notices that OF is on, so it proceeds to print the heading lines you have conveniently conditioned on OF. OF is then turned off. The next time a real page overflow occurs when that Fetch Overflow Detail (or Except, or Total) line shows up, the headings are printed again.

What is beautiful about this process is that it is almost invisible. There are no calculations telling the program to print headings. Except for that *INZSR line of code, there are no lines of code that set on the OF indicator. The indicator is (except at the start) controlled entirely by the system. No calculating of total lines printed on the page. If  there is some chance of there not being any detail lines to print, yet you still want headings, you can generate at the end an Except line with Fetch overflow that says “No records read” - or even a blank line. In either case, it will detect the OF you set on and print the headings before printing the line of desired detail output.

I have not noticed any feature like this in any other general-purpose programming language - but then RPG is NOT a general-purpose programming  language. As I have said in earlier posts, it is designed to push data around, update files and print reports. Input, process, output.

Quite frankly, I think Fetch Overflow  is the kind of feature that writers of RPG should be proud of. Very slick. More recent applications of the language have tended to shy away from emphasis on the report generation facility of RPG, but that should not detract from our recognition of the terse elegance and power of RPG.

For some geek entertainment (for programmers), try this weblink to a website for RPG programmers and read some entries. Go to the RPG archive and enter the search term “cycle” or “matching record” and see what kind of discussions (arguments?) you find. It is a fine way of showing that, while the “old” techniques are in decline, they are still very much alive. It is amazing how much of the negative press regarding old programming techniques stem from sheer ignorance.

Programming, though done with the idea of controlling computers, is of necessity reflective of the biases of humans. Though RPG performs well the functions for which it was designed, it will never please everyone. If it had not had the backing of IBM and didn’t run on hardware that was rock-solid (from IBM, of course), it might not have been proliferated as much as it has. It had deficiencies (string handling being one of the worst) that would have resulted in its being discarded if its superior file-handling capabilities had not taken priority and a body of machine-specific software programs had not already been written for the IBM midrange (System3/System 32/System 34/System 36/System 38/AS400). Other companies created their own RPG compilers, but those never did catch on with business customers.

This IBM exclusivity, though, has hurt RPG. The machine it runs on currently (iSeries, or whatever IBM is calling it today) is viewed as ancient technology, even though on performance it blows the doors off any credible competitior. For reliability, to talk about reliability of iSeries versus the Wintel machines is a joke. But the Wintel boxes are cheaper, and IBM has not been able to put across the cost-benefit ratio of iSeries, which runs with the barest minimum of babysitting, versus Wintel or Unix or Linux with their relatively cheap boxes to go with an army of systems analysts and database administrators.

As a result, IBM is now pushing iSeries (system i?) as a server capable of handling multiple operating systems, which of course it does quite capably. But as a result, the body of work done for the iSeries and its forebears is in danger of being set aside. RPG is now perfectly capable of communicating with the Internet and handling Internet processing-  but the little ignorant children who find Java easy and RPG difficult (go figure) are in danger of letting RPG be discarded via attrition. RPG has an active user community- but whether or not it has sufficient clout with the people with the money to make the facts known is somewhat less clear. We hope that they do, but whether they will before the RPG graybeards like me retire is problematical.

Perhaps there is a silver lining, though- maybe we old guys can make a killing as part-time consultants after retirement, maintaining the systems we helped to create. It will take time to rewrite hundreds of millions of lines of RPG code in RPGIV (even using the CVTRPGSRC utility to do it) or in some other language inferior for the purpose of business software creation. The new recruits may find those systems too old and the Chaucerian RPG dialect too incomprehensible. Sad.