RPG and Programming

I didn’t think I’d be considering free-format RPG so soon as a final solution for the current RPG programs on the system at work (some of which date back to the 70’s), but a number of interesting events have taken place.
Our IBM representative made us aware of a software package that accomplishes the conversion of RPGII, RPGIII, and RPG400 programs into true RPGIV. This means (among other things) ditching the left-hand indicators and attempting to really use more modern operation codes. If you are at all familiar with what would be necessary, you can appreciate how difficult this task would be without some kind of automation. The painstaking code changes and testing would be incredible on a code base of any size.
This package is called RPG Toolbox, from Linoma Software.
So we downloaded the trial version, which allows 10 source conversions from another RPG dialect - or even RPGIV that has simply been converted from an older format using IBM’s CVTRPGSRC utility. CVTRPGSRC simply reformats old code into RPGIV syntax; it does not attempt to modernize it. We converted a source member with a lot of left-hand indicators, and we were impressed how it converted into a reasonably neat format without those indicators. What really impressed me was that it would attempt to convert a series of MOVE statements, which was commonly the method used to piece together larger fields, into EVAL statements. This could be a dangerous technique, but the RPG Toolbox utility carefully analyzes the code to make sure it is safe. In the manual, it describes how it makes decisions about code conversion; if one is interested in this product, that person would do well to download and read the manual before making decisions about what the program should do in the course of conversion.
I also had an inspiration- why not use it to convert RPG IV to freeform? In earlier posts, I have declared my reservations about how valuable freeform is. But, if it’s easy, why not live dangerously? So I tried converting one of the RPGIV programs in my file cross-reference system to freeform. The results were impressive. It looked VERY good. The more I looked at it, the more I liked it. So I said to myself, why not convert ALL our code to freeform?
But then my younger cohort brought me back down to earth. He wasn’t so sure that would be a good idea. So we converted the old-format code referred to above into freeform. The results were messy. There was much code that simply could not be converted, often involving the MOVE and MOVEL opcodes. (I have also commented on this previously.) When this happens, the free-format code is enclosed in a /FREE - /END-FREE set of compiler directives and the code drops back into fixed format. Switching back and forth between free and fixed can be maddeningly ugly.
So I have backed off the idea of a total conversion to freeform RPG. But we still feel that converting our code to a clean RPGIV that uses as much as possible of RPGIV’s modern syntax and opcodes would certainly be a good thing; and it appears that it would be relatively safe. We have ordered the software. Where the conversion of a program to freeform would not create too messy a result, we can still convert it.
I was also surprised to learn how quickly the most experienced one on our crew would take to freeform. He first started programming in ALGOL; so freeform is not really that much of a leap for him. He has in the past several weeks written new programs in freeform RPG, of his own free will. My other partner has a Pascal/Java background; so freeform would come “naturally” to him. I am the one who will take some getting used to the idea.

Subfile processing is an integral part of RPGIV data entry and data display. To be a real pro, you need to handle them properly. There are any number of ways to build and maintain them. Depending upon your level of expertise, you can do really sophisticated data processing tasks with subfiles. (Yes, I know that phrase data processing is so 1970’s.)

Dealing with them at my place of employment is an ongoing task. My two fellow programmers, though having much experience in programming, are relatively new to RPGIV. To help enhance their skills, one of them bought the excellent book by Kevin Vandever, “Subfiles in RPGIV”.

One of them was using it to help him build the first subfile program he had ever written from scratch. And wouldn’t you know, his techniques were different from mine! And the author had used my technique when he first started programming subfiles, but he had found a better one. Oh, my.

For the uninitiated, subfiles are a way to handle update and display of data files, by allowing you to display multiple instances of data in (usually) a columnar format, then treat each line as a separate data record. In RPGII, subfiles were not available; to process it, you had to plug your file data into arrays of information, each data field in an individual record linked by being the same array element in multiple arrays.

Anyway, the author recommends a technique that is clean and efficient. He uses the DDS keyword SFLNXTCHG to allow the programmer to flag for further processing data in a record that has been changed. The operating system keeps track of changes, and this allows the programmer to process only those subfile records that have been changed by the programmer. This is done by using the READC (Read Changed record) RPG opcode. Using SFLNXTCHG with an indicator allows the programmer to mark a line of data as “changed” when it processes the second time around. If one entered erroneous data in error, and the program would not accept it, the data would still be marked as “changed” and the record would be read the next time around by the READC opcode, even if the data entry person did not correct or change the data.

My approach, the way Mr. Vandever set aside, is to keep track of the number of records (entries) in the subfile. To process the subfile, I CHAIN to each subfile record by relative record number, whether it is changed or not, and process the data as needed. I can do this as many times as desired.

Why do I do it my way? Because I’m an ignorant so-and-so, I guess- or at least I was when I was learning many years ago how to program subfiles. I vaguely remember attempting to use READC to process only changed subfile records. However, I was not aware of the SFLNXTCHG keyword; and when I would read a subfile record and find an error, I couldn’t see how to tell the system to process the record the next time around. Suppose the screen had five subfile records displayed. I changed two records, and one of them was in error. Once I fixed the one in error and the program looped through with the READC opcode again, it would not pick up the correct one from the previous pass because it couldn’t tell it has been changed! When I saw an example of the CHAIN technique, I realized that it would always work, so I stuck with that technique.

So do I change now to the more efficient technique in the future? Wellll…. probably not. Being a creature of habit, for one thing; and the fact that changing my technique will not gain me all that much and would make the “read subfile” task more complex.

But what about all the unnecessary CHAIN operations, which on a large subfile mean an awful lot more subfile accesses than necessary? If you have read my blog from the start, you might remember when I told about a hissy fit  I had when I worked on a program where a previous programmer read a detail data file by CHAIN instead of READ and increased the processing time by a factor of 10 – from 15 minutes to 2.5 hours.

I suppose that I have been corrupted by the greased lightning bottled in the latest machines. In my file cross-reference program, I load source files into subfiles so that you can drill down into the subfile and find out file characteristics, even view the data in the file by putting the cursor on a file name and clicking a function key. When I tell the system to display source, it can put 9,999 lines of source to the subfile in less than a second. Why sweat the difference in processing one record with READC versus, say, 12 when using CHAIN? In a large installation with hundreds of users on a slower machine, perhaps it would be a concern- but not now. My concerns about performance are pragmatic, not ideological. As I have said before (as a matter of fact, in that previous blog post I mentioned above), I don’t sweat over nanoseconds.

Also, at least with reference to my cross-reference program, using READC is irrelevant. I do not change the subfiles in any way. They are simply a jumping-off point for further processing. I use EXFMT (which means write the format, then accept input) to display them, but only to allow the input of a right-click or a command key or a search argument to use with a command key.

A minor quibble would also be that an edit that relied on READC would catch only errors that had been introduced by the current data entry operator. There is no way to guarantee that the data in the subfile is error-free when it is first displayed, especially if the data is also updated by other programs. If you edit every line, you can check for errors in all the data, including preexisting errors.

So my partners are free to use the new techniques, but I am not likely to choose to do so in the future.