RPG and Programming

In a blog post on my son’s blog, I ruminated on helping young people to learn to program and what might be a good place to begin learning programming. At the end of the article, I suggested Oberon-2, a language created by Niklaus Wirth (creator of Pascal and Modula-2) and an associate. At the same time, I bemoaned the fact that I seemed to be unable to learn new languages, perhaps because of burnout or fatigue, perhaps lack of time, or even lack of a suitable project.

I have just about concluded that it is time to put up or shut up. I have a compiler, I have a book, and I have a project. It is time for me to go ahead and exercise some initiative and learn the language.

But perhaps you wonder why Oberon-2? Why not Java, which is also on the AS/400? I have commented on the steep learning curve. I have attempted Java tutorials; I have found them to be of a most forbidding nature. I don’t know if Java is supposed to be a “small” language or a “large” one; all I know is that one look at the documentation (see how long it takes you to scroll through the class list) makes me want to cringe. Of course, with any language, you usually use only subset of a language in your day-to-day programming; the problem here is finding what your subset consists of.

Why Oberon-2? It is object-oriented programming without the gobbledygook. A “method” is a procedure. A “class” is a record type. A variable of a record is an “object” or an “instance” of a “class”. A procedure call is essentially the same as “passing a message to an object”. I find the unquoted terms in the previous sentences more comprehensible than the corresponding terminology (in quotes) that has been perpetuated for the Java and C++ and Smalltalk environment. And it does not appear that it will take that long to learn how to write useful code; that is important for me.

So I propose to begin learning this language, and from time to time report on my progress. Knowing that at least a few people might be observing may also exert a certain social pressure to press on.

Fetch Overflow is one of those somewhat obscure programming constructs that a “modern” programming language lover would likely scorn or ignore, but it epitomizes what makes a programming language like RPG special.

Fetch Overflow requires Output specifications, which are being used less and less, and it requires the use of an overflow INDICATOR (aaaaaggghhh! See the Java programmer running away into the bushes.) Some RPG programmers may not even know how to use it; more’s the pity.

For those applications that continue to generate reports on real paper, the problem of handling end-of-page conditions, including when to print headings, has always been an issue. RPG’s use of the overflow indicator (which may be OA-OG or OV - OVerflow, get it?) has always been to me a very elegant way to generate proper report appearance.    You can condition lines of print on the indicator, if you use O specs. The system default for printing is a 66 line page, with overflow at line 60; but this can be adjusted to suit the needs of the application. Even with an externally defined print file, you can define a numeric overflow indicator and print lines and do jumps to new pages at page overflow.

There is a column (I believe it is 15 on the standard RPG specification,something different on the RPG IV O spec), where you tell whether it is (D)etail, (H)eading, (T)otal, or (E)xception time output. The next space to the right is where you can put an F, for Fetch Overflow. When this occurs, the programmer is assuming that there will also be some other lines conditioned on the overflow indicator, usually specifications that tell the program to advance to the top of the next page and print defined headings, often several lines of them. If the printing on the page has reached up to or past the overflow line (let’s say line 60), and that line meets the requirements for being printed, the program will print all the lines conditioned on the overflow indicator (let’s say OF), then print the line that fetched the headings, then set off OF.

In the past, often the OF indicator would be used in conjunction with the 1P (first page) indicator. But there is available an even slicker way of handling the headings. Use an *INZSR subroutine, which is automatically executed before any files are read. In that subroutine, insert   SETON   OF   or    EVAL *INOF=*ON.

Now, what happens when the first detail line is read with the Fetch Overflow flag? It notices that OF is on, so it proceeds to print the heading lines you have conveniently conditioned on OF. OF is then turned off. The next time a real page overflow occurs when that Fetch Overflow Detail (or Except, or Total) line shows up, the headings are printed again.

What is beautiful about this process is that it is almost invisible. There are no calculations telling the program to print headings. Except for that *INZSR line of code, there are no lines of code that set on the OF indicator. The indicator is (except at the start) controlled entirely by the system. No calculating of total lines printed on the page. If  there is some chance of there not being any detail lines to print, yet you still want headings, you can generate at the end an Except line with Fetch overflow that says “No records read” - or even a blank line. In either case, it will detect the OF you set on and print the headings before printing the line of desired detail output.

I have not noticed any feature like this in any other general-purpose programming language - but then RPG is NOT a general-purpose programming  language. As I have said in earlier posts, it is designed to push data around, update files and print reports. Input, process, output.

Quite frankly, I think Fetch Overflow  is the kind of feature that writers of RPG should be proud of. Very slick. More recent applications of the language have tended to shy away from emphasis on the report generation facility of RPG, but that should not detract from our recognition of the terse elegance and power of RPG.

For some geek entertainment (for programmers), try this weblink to a website for RPG programmers and read some entries. Go to the RPG archive and enter the search term “cycle” or “matching record” and see what kind of discussions (arguments?) you find. It is a fine way of showing that, while the “old” techniques are in decline, they are still very much alive. It is amazing how much of the negative press regarding old programming techniques stem from sheer ignorance.

Programming, though done with the idea of controlling computers, is of necessity reflective of the biases of humans. Though RPG performs well the functions for which it was designed, it will never please everyone. If it had not had the backing of IBM and didn’t run on hardware that was rock-solid (from IBM, of course), it might not have been proliferated as much as it has. It had deficiencies (string handling being one of the worst) that would have resulted in its being discarded if its superior file-handling capabilities had not taken priority and a body of machine-specific software programs had not already been written for the IBM midrange (System3/System 32/System 34/System 36/System 38/AS400). Other companies created their own RPG compilers, but those never did catch on with business customers.

This IBM exclusivity, though, has hurt RPG. The machine it runs on currently (iSeries, or whatever IBM is calling it today) is viewed as ancient technology, even though on performance it blows the doors off any credible competitior. For reliability, to talk about reliability of iSeries versus the Wintel machines is a joke. But the Wintel boxes are cheaper, and IBM has not been able to put across the cost-benefit ratio of iSeries, which runs with the barest minimum of babysitting, versus Wintel or Unix or Linux with their relatively cheap boxes to go with an army of systems analysts and database administrators.

As a result, IBM is now pushing iSeries (system i?) as a server capable of handling multiple operating systems, which of course it does quite capably. But as a result, the body of work done for the iSeries and its forebears is in danger of being set aside. RPG is now perfectly capable of communicating with the Internet and handling Internet processing-  but the little ignorant children who find Java easy and RPG difficult (go figure) are in danger of letting RPG be discarded via attrition. RPG has an active user community- but whether or not it has sufficient clout with the people with the money to make the facts known is somewhat less clear. We hope that they do, but whether they will before the RPG graybeards like me retire is problematical.

Perhaps there is a silver lining, though- maybe we old guys can make a killing as part-time consultants after retirement, maintaining the systems we helped to create. It will take time to rewrite hundreds of millions of lines of RPG code in RPGIV (even using the CVTRPGSRC utility to do it) or in some other language inferior for the purpose of business software creation. The new recruits may find those systems too old and the Chaucerian RPG dialect too incomprehensible. Sad.

One of the more interesting things about freeform RPG, at least currently, is its support of the MOVE/MOVEL RPG operation codes. It doesn’t.
For those not so familiar with RPG, MOVE (move right) and MOVEL (move left) are codes that you will find in almost any standard RPG program. What do you do with them?
You can:
1. Move a string into another string. If the target string is smaller, the move ends with the filling of the string. For example, assuming the field MONTH as a 9-long string with value “NOVEMBER” and a 5-long string RESULT:  MOVE MONTH RESULT results in ‘MBER ‘ ; MOVEL MONTH RESULT result is ‘NOVEM’. If the smaller field is MOVEd to the larger, what is left remains unchanged (unless the move is given a code (p) which causes it to be filled with blanks). If RESULT was ‘NOVEM’, MOVEL ‘123′ RESULT would produce ‘123EM’
2. You can MOVE strings to numeric fields, and strings to numbers. In this case, the system will try to convert the string to numbers; this is no problem, especially if the string is already a number.
3. Move strings and numbers to DATE type fields (which RPG can manipulate with other techniques, such as adding a month to the date), and vice versa.
In fact, 14 pages in the RPGIV manual is devoted to examples of how to use the MOVE opcode. (As we said above, MOVEL just starts from the left. Almost as many pages are given to MOVEL).
You would think at first that IBM would want to use this code, in view of its widespread use and of its power. Syntax would not be an issue; I deliberately did not attempt to put my example above in fixed format, just to show how obvious the syntax was.
But, what does the IBM manual say about MOVE? To quote:
“Free-Form Syntax: (not allowed - use the EVAL or EVALR operations, or built-in functions such as
%CHAR, %DATE, %DEC , %DECH, %GRAPH, %INT, %INTH, %TIME,
%TIMESTAMP , %UCS2, %UNS, or %UNSH )”.
To this list I would add %SUBST(substring), at least.
I can think of no practical reason why the implementers did not simply allow MOVE into freeform. Perhaps it’s because I’m just an ignorant junior college grad, from when they didn’t even have CompSci courses in JC. Since I am not a computer science major, to me the implementation would be a snap; when the compiler sees “move field1 field2;” in freeform, do exactly the same thing you would if you saw MOVE FIELD1 FIELD2 in fixed format. (Copy the compiler code! What a revolutionary idea!)
I, for one, think the problem is ideological. MOVE (along with its cousins) is too powerful a code. Since high school grads (like me) know how to use it, it obviously is too difficult for CompSci graduates to understand; you must replace it with another opcode (EVAL) and one or more of at least 12 BIFs listed above. Makes it easier. Yeah, right. We must make it simple so the ignorant Java programmers can understand. Pardon me while I get ticked off by intellectual condescension. The compiler writers are evidently trying to pry us away from an operation they feel is somehow inferior. Never mind that it works reliably, precisely, and elegantly.
Never mind that you have to be very careful how you use the BIFs, since you might not end up with the exactly equivalent result. You wouldn’t have to bother if you didn’t change it in the first place. Multiply this by the millions upon millions of MOVEs and MOVELs that likely exist in existing code, and you begin to see the problem.
The code will almost certainly be longer and likely more complicated if you attempt to convert the code. You would think brevity would be prized, since it seems such a virtue when it goes the other way, when they come up with a BIF that seems to do the job more succinctly than its fixed-format equivalent. Apparently here, they are not shooting for brevity; and I find it hard to see how they enhance the clarity of the code by avoiding MOVE.
Frankly, if they are trying to promote the use of freeform, I think they are shooting themselves in the foot by not implementing MOVE. From what I have read on the RPG forums, the lack of MOVE support stands in the way of easy freeform conversion. The fixed-to-freeform converter cannot handle it, so a /END-FREE must precede it, dropping the code back into fixed-format. Many other opcodes can be handled more or less elegantly using BIFs; not using MOVE in freeform often ends up forcing the programmer to use some Rube Goldberg BIF to try to salvage the situation.

I can only wonder why they are taking an ideological stand on this subject. “Dumb” is the nicest word I can think of with reference to their decision.

It has been too long since my last post. What’s going on? Well, for one thing, I’ve done a couple of articles for work.com (a bit of shameless self-promotion here) , on “recovering data from a dead computer“  and another on “natural language processing” - text-to-speech and speech-to-text software.  I’m actually getting paid for my writing; and though I’m getting paid less than minimum wage (on an hourly basis) for doing it, it really is neat for your writing ability to be recognized.

But I haven’t really been all that busy writing those things. It’s kind of hard to explain. I think the problem is that I have convinced myself that everything I write must be a work of art, with detailed logic, clever phrases, and quotations. After the first flush of enthusiasm for the blog wore off, I found it not so easy to write through the brain fog that envelops your mind when you are tired at the end of the day. I have the energy to read my e-mail, surf my favorite web sites, but not enough to sustain several hours of determined composition.

But I should know better. Over and over again through the years I have read that the best way to write is to just sit down and do it. If you don’t feel like two hours of writing, do 15 minutes. That’s why this program I work with has a draft mode- what I write doesn’t immediately have to go to the Web. Just do it.

So I will try to do better. Just getting a new computer at work generates some enthusiasm. Things that took prohibitively long before now can be done so quickly. As a result, I have been enhancing and cleaning up the program and file cross-reference program I have written at work. I can track, in a nested fashion, what programs call what programs; and within a program I can click on a file name and get a file description or even display the file itself, scan the text for strings, and other neat stuff.  When it took well over an hour to generate the files necessary for the program, and now the files can be generated in under two minutes- that just gives you more of a feeling of power.

I was receiving the Web statistics on my various posts, and I found it very interesting that the articles that seemed to generate the most interest involved, in some fashion, freeform RPG. Even though I myself am not a big fan of it, it may be interesting to pursue that topic further in future posts, perhaps in a less dismissive way. I still believe in the validity of my arguments, but it may be worthwhile to analyze it to see why people are intrigued by it.

Before getting back to technical topics, I thought it worth a few lines to consider the programming environment and how it relates to programming technique.

When I was starting to program, the IBM Midrange (with the System/34 as its current representative) was pointed at small business. Its programming language, RPGII, was not viewed as a highly sophisticated tool. People without a Computer Science background could be expected to learn it without a great deal of fanfare. A common way to start was to begin as a computer operator and work your way up.

I suspect that those days are gone forever. The days when you would give the operator a manual and some programs to read are likely done. There are high expectations for programmers these days; if there are entry-level positions, they are well hidden; and those are probably reserved for those with at least some background in college-level courses.

The sophistication of the code also at times left something to be desired. More programmers were needed than could be generated by college, and so their techniques were not always clear. That was especially true with RPG II, which did not have structured operators (IF - ENDIF, etc.). Code was produced of the sort that positively gives me a headache today. A headache to maintain- but it worked.

And so we read today of sophisticated techniques derived from our study of other programming languages. Some are good, others not so good. But more and more sophistication is being demanded. And that can be a good thing.

But is zeal to pursue all these techniques a good thing? As always, the answer is, not always. If your needs are sophisticated, by all means, keep on pushing. But what if you don’t need these techniques? What if the business you support is profitable without them? Is it really a bad thing if you are not an early adopter?

The place I work now fits those criteria. It has been on the IBM midrange for over 30 years. Some of the programs are that old, too. Do you scrap code just because it’s “old”? No. You may try to improve it as time is available and the need arises, but first of all the business must be supported effectively. It would be a waste of time to try to shoehorn a new technique involving user spaces and message queues when it simply is not necessary. If some new technology will demonstrably advance the business, we get it.

So my job is not to be an early adopter, but to support the business, make improvements where I can, but not feel deprived if I am unable to employ some sexy new subroutine or program calling technique. The geek in me wants to try everything I read in the various technical forums, but the pragmatist in me tells me that I cannot do it. We just got a new machine with 13 times the space and over 20 times the speed of the old one; that ought to keep me interested for quite a while. We are connected to the Internet, but not directly; so many of the fancy new features of RPG that allow direct connect to the Internet simply are not needed. My co-worker does not have my RPG background, but I think he knows a lot and is learning more; his programming abilities go beyond RPG to the point that I can only envy them, not emulate them; I have neither the time nor the energy to do so. But his energy energizes me.

Updating code calls for patience. That is a virtue. I am not being paid so I can be entertained; I am being paid to create a positive business result for my employer; as long as I am constructive and keep on learning, I will be happy. I am not a small piece of a big puzzle; I am a big piece of a small one.

Years ago, to talk about what language to use when you were going to write a program would be somewhat silly. You would use the language that came with the computer. Often, you would only have one language to work with - COBOL, RPG, BASIC, PL/I, C, or whatever. As microcomputers came on the scene, you still had few options beyond BASIC and Pascal, due to the small amounts of memory available. This, of course, changed as time went on, but if a shop had a particular language they worked with, they generally wanted to use that language only.

Being an old baby-boomer brings one a certain perspective to programming. Some my age started way back in the 60’s with their COBOL, assembler, and Fortran. Back in the 60’s, I had never knowingly come within miles of an actual computer. And being of a distinctly lower-class (economically, that is) background, I found computers interesting, but there was little chance of them becoming  the center of my life. Of course, the only lives centered around computers then were the programmers and manufacturers of them. Now, of course, anyone who wants to can program one. Even doing an Excel spreadsheet is programming of a sort.

But is being a programmer automatically a good thing? I would submit that whether you are a good programmer is determined by more than your ability to program. I believe it is also determined by what you program.

Now, we will grant that being a programmer of business programs is not the most exciting of occupations. But at least the program, if it is written to perform effectively in its context, may be considered to perform a useful service. That is what programs were originally designed to do. (Some might quibble that ENIAC, arguably the first electronic computer, was prompted by the need to perform calculations of artillery shell trajectories; it was useful, but some would point it out as another use of science in the service of warfare (see atomic bomb), which they would consider a dubious use of science.)

But is all programming good? I submit that the answer is no, even if it performs as designed. Back in 1982 there was a game called “Custer’s Revenge”. The point of the game was a naked General Custer “running across a desert obstacle course to reach and ravage an Indian maiden”. (August 30, 1982, issue of Advertising Age) Was there any redeeming social value in that game? I think not. Since then the computer graphics have improved, but from various news reports and reviews of computer games, I have concluded that the content of many games has not improved - only deteriorated, along with the general state of society. Sadly programmers, for all their cleverness and intelligence, have contributed to the moral decline of society in general.

And the programming need not be lascivious in nature to have a negative effect. Gratuitous violence hardens young people to the significance of the real thing, the hurtfulness and pain that violence in reality causes. Also,  many video games are designed to be played for hours on end, sometimes over a period of days. Is this the kind of thing that young people and adolescents need? About the only time such a self-centered pursuit would be good would be if the neighborhood was so bad that it would be dangerous to go outside.

Perhaps that is why I cringe a little when I realize that there are college degrees in videogame programming, here for example. There may be good uses for video games, but one suspects that educational video games are not where the big bucks are. How many young programmers will go into video game programming because of the good it will do for mankind? Probably not many.

Hopefully, if you are starting to program now, you want to learn for the fun of it and perhaps the intellectual challenge. Once you gain the skills, for the good of all - including yourself- be careful about what you choose to program.

I admit it. I am beginning to get a case of writer’s block. What do I write about next?

I’ve been saving this subject for a while, but I could no longer resist using this headline. What does it have to do with programming?

It’s quite possible that even some RPG programmers are not aware that that you can FORCE RPG calculations. According to the RPG II manual, “The FORCE operation allows selection of the file from which the next record is read. The FORCE operation can be used for primary and secondary input and update files… Factor 2 in a FORCE operation identifies the the file from which the next record is read.”

The first time I ever saw this opcode used was in 1984, five years after I started programming. It was also the last time. I have NEVER seen it used in a working piece of code since then. My son’s textbook “RPGII, RPGIII, and RPG/400 with Business Applications” by Stanley E. Myers does not even show it in its index.

I don’t even remember how the code was used. I knew at the time. I examined the calculations to determine how the program worked, why it used the FORCE opcode, made the necessary changes, and went on with my life.

What makes this interesting is that you never hear a cry for the abandonment of the FORCE opcode. It is obviously centered around the RPG cycle, which of course is the spawn of Satan, and no self-respecting RPG programmer wants anything to do with it. “READ your file, don’t reCYCLE it,” is their battle cry. Yet FORCE escapes their notice. It is still in the RPGIV manual, so some nefarious, wizened old RPGII programmer (or original RPG programmer from the 1960’s!) might see a place where he could use it, and give new RPGIV programmers yet another reason to run away from their ILE terminals in panic at the sight. “I don’t understand…I don’t understand!!!!”

Or… NOT. If a real programmer sees an opcode he doesn’t understand, what will he do? He will look in the manual and find out what it’s all about. If he thinks it’s interesting he may say, “That’s neat” (or whatever catchphrase they use today to express approval), and perhaps thereafter see if he could use it too, perhaps finding out how it is best used. If not, he may ask, “Why would anyone use that opcode? Is that dumb or what?” (Choose your own phrase of disapproval.)

Anyone who warns “Don’t use the cycle”, or “Never use indicators”, or “Never use level breaks” are really insulting the intelligence of the ordinary RPG programmer. They are assuming that the grunt is too stupid to understand the evil he is getting himself into. If they then say that “modern” RPG programmers don’t do this or that, it becomes an ad hominem attack rather than a logical one. And if they say that new programmers won’t be able to understand this or that piece of code because of the technique or opcode used, they are wasting their breath.

New programmers will not reach out and accept RPG if it looks like Java or Pascal or C; they will just keep their Java or Pascal or C. RPG does not have to change its appearance in order to be successful, anymore than COBOL does. COBOL is still around and making itself available across the mainframe, midrange, and PC world. It has its weird keywords, too. If a piece of code is not wise to use, programmers will discover it- and avoid it.

So, when you’re programming, lighten up. Don’t worry, be happy. May the FORCE be with you. It won’t hurt you if you never use it.

One rule repeated in almost every discussion of programming is, “Don’t re-invent the wheel.”
The idea is, if someone has written a program, routine, or system to solve a particular programming or business problem, it is foolish to waste programmer time and other resources writing another routine to accomplish the same thing. With RPG now more capable of working in a mixed-programming language environment, the suggestion now often is, if a program in C solves a programming problem, it is better to call the C routine, by whatever means available, than to insist on writing it in RPG.
As a concept, I think it makes sense. At least superficially, it seems pointless to duplicate someone else’s effort. Since often you don’t know the identity of the person who wrote the original routine, I like to call the practice “Depending on the Kindness of Strangers”, after the famous line from Tennessee Williams’ play “A Streetcar named Desire”.

But when it comes down to the implementation of the practice, I sometimes think that depending on the kindness of strangers in programming is about as good an idea as it was for Blanche Dubois - in other words, not very good.

Read the rest of this entry »