Remove gcdiff and adjust testsuite

[ddeclerck]

This PR removes gcdiff and adjusts the testsuite to accomodate for the changes.

[GitMensch]

That approach is good, but let's try to get rid of those sed invocations (or make them simpler/less used, for example with SOURCE_DATA_EPOCH) in 3.x first, then merge the change over (I think most of the effected lines will use gcdiff in 4.x).

Oh, and another note: you don't need to use the underscore-sed hack, it may be much easier to use [[some data ]] instead of the current [some data _] (I just learned that too late, and seemingly haven't adjusted all the tests that expect trailing spaces.

Interesting to see that this PR does uncover some tests that have a different result and was not recognized before because of the use of gcdiff.

[ddeclerck]

That approach is good, but let's try to get rid of those sed invocations (or make them simpler/less used, for example with SOURCE_DATA_EPOCH) in 3.x first, then merge the change over (I think most of the effected lines will use gcdiff in 4.x).

I'd do that when time permits.

Oh, and another note: you don't need to use the underscore-sed hack, it may be much easier to use [[some data ]] instead of the current [some data _] (I just learned that too late, and seemingly haven't adjusted all the tests that expect trailing spaces.

Ah, I though those underscores were just a visual reminder that the trailing spaces are intentional.

Interesting to see that this PR does uncover some tests that have a different result and was not recognized before because of the use of gcdiff.

I was as surprised as you are.

[GitMensch]

... waits for rebase ...
in any case please also update the references: NOTES + TODO + AUTHORS (side note: feel free to open a PR against 3.x for adding OCamlPro / the contributors, if you see any elements missing) and add Changelog entries for all of the three sub-folders [bin + build_windows may need the (C)2025 update in the footer when doing so].

[ddeclerck]

Rebased. Still a bit of work to do.

One test in syn_file.at (XFD directive and creation) reveals that the function output_xfd_file (sqlxfdgen.c) does not use SOURCE_DATE_EPOCH. Should it do so ?

in any case please also update the references: NOTES + TODO + AUTHORS

What should be changed in NOTES ?

[GitMensch]

NOTES is a document we'll handle berfore handling the TODO - you can just delete any elements related to gcdiff in this file as part of this PR (less to glance over).

And yes: SOURCE_DATE_EPOCH must be handled in everything that is generated by the compiler by using the central general time routines that are used otherwise within cobc for timestamps; it may should use the same time than the generated .c file, but that's up to the fixing person (= you :-).
Fixing SOURCE_DATE_EPOCH and the syn_file.at should really be a separate commit - feel free to have both commits (cleanup + fix output_xfd_file) in this PR, or split into two PRs - as long as it ends up with two commits in trunk upstream I don't mind.

[GitMensch]

both commits are nearly fine, please adjust (and feel free to do that in the upstream commit directly, if you want to)

[ddeclerck]

Merged in SVN @ 5526 and 5527.

[GitMensch]

Test failures on 32bit - so we either need a sed script or a conditional test here:

846. run_misc.at:9704: testing Test dump feature (4) ...
./run_misc.at:9997: SOURCE_DATE_EPOCH=1011960000 \
$COMPILE -debug -fdump=ALL prog.cob
./run_misc.at:10000: COB_DUMP_FILE=dumpall.txt \
$COBCRUN_DIRECT ./prog
./run_misc.at:10150: $SED -e 's/Started by .*prog.exe/Started by .\/prog/g' \
dumpall.txt > dumpall.sed
./run_misc.at:10152: $DIFF reference dumpall.sed
--- /dev/null	2025-05-21 13:01:42.412383655 +0000
+++ /home/appveyor/projects/gnucobol-trunk/tests/testsuite.dir/at-groups/846/stdout	2025-05-21 13:10:30.074137726 +0000
@@ -0,0 +1,4 @@
+17c17
+< 01        PTR                             0x0000000003404d71
+---
+> 01        PTR                             0x03404d71
./run_misc.at:10152: exit code was 1, expected 0
./dumpall.txt:
> 
> Module dump due to signal SIGSEGV
> 
>  Last statement of "sub2" was MOVE at line 4 of cpyabrt
> 	ENTRY sub2 at prog.cob:238
>  Last statement of "prog" was CALL
> 	CALL-IT-OMIT at prog.cob:135
> 	MAIN-100 at prog.cob:99
> 	ENTRY prog at prog.cob:91
>  Started by ./prog
> 
> Dump Program-Id sub2 from prog.cob compiled Jan 25 2002 12:00:00
> 
> WORKING-STORAGE
> **********************
> 01        ZRO                             000000000
> 01        PTR                             0x03404d71
> 01        HEXV                            013
> 01        IDX                             000000000
> 01        TSTREC.
>   05      TSTDEP                         'XXX'
>   05      TSTX (1).
>        15 TSTG-1 (1)                      01
>        15 TSTX-2 (1,1)                   'A'
>        15 TSTX-2 (1,2)                   'XX'
>        15 TSTX-2 (1,3)                   ALL SPACES
>        15 TSTX-2 (1,4)                   'xx'
>   05      TSTX (2).
>        15 TSTG-1 (2)                      02
>        15 TSTX-2 (2,1)                   'B'
>        15 TSTX-2 (2,2)                   'XX'
>        15 TSTX-2 (2,3)                   ALL LOW-VALUES
>        15 TSTX-2 (2,4)                   'yy'
>   05      TSTX (3).
>        15 TSTG-1 (3)                      03
>        15 TSTX-2 (3,1)                   'C'
>        15 TSTX-2 (3,2)                   'XX'
>        15 TSTX-2 (3,3)                   ALL HIGH-VALUES
>        15 TSTX-2 (3,4)                   'zz'
>   05      TSTX (4).
>        15 TSTG-1 (4)                     ALL HIGH-VALUES
>        15 TSTX-2 (4,1)                   ALL HIGH-VALUES
>        15 TSTX-2 (4,2..4) same as (1)
>   05      TSTTAIL1                        X  
>                                      1 x 5811
>   05      TSTCOMP3                        58585
>   05      TSTLONG                        'Quick brown fox jumped over the dog              Quick br'
>                                       57:'own fox jumped over the dog'
>   05      TSTHEX                          Q u i c  k e r    g r e y    f o x    j u m  p e d  
>                                      1 x 51756963 6B657220 67726579 20666F78 206A756D 70656420
>                                           t h e    c o u g  a r   X  X X   X  X X X X  X X   X
>                                     25 x 74686520 636F7567 61722058 58581158 58585858 58580758
>                                           X X X X  X X X X  X X   X  X X X X  X X X X  X X X X
>                                     49 x 58585858 58585858 58580D58 58585858 58585858 58585858
>                                           X X X X  X X X X  X X X X  X X X X  X X X X  X X X X
>                                     73 x 58585858 58585858 58585858 58585858 58585858 58585858
>                                           X X X X
>                                     97 x 58585858
>   05      TSTHEX2                        XXXXXXXXX\0XXXXXXXXXXXXXXXXXXXXX\\XXXXXX\0XXXXXXX\tXXXXXX
>                                     54 : AXXXX\rX
>   05      TSTTAILX                       'Quick brown fox jumped over the dog     '
>                                   trailing LOW-VALUES
> 
> LINKAGE
> **********************
> 01        X                               0757.99
> 01        TSP-RECORD                     
>  >>>> Dump of TSP-RECORD aborted! <<<< !!
> **********************
> 
> END OF DUMP - sub2
> **********************
> 
> Dump Program-Id prog from prog.cob compiled Jan 25 2002 12:00:00
> 
> FD FLATFILE
> **********************
>    File is OPEN
>    FILE STATUS  '00'
> 01        TSPFL-RECORD.
>      10   CM-CUST-NUM                    'ALP00000'
>      10   CM-COMPANY                     'ALPHA ELECTRICAL CO. LTD.'
>      10   CM-DISK                        '8417'
>      10   CM-NO-TERMINALS                 0010
>      10   XX-CUST-NUM                    'ALP00000'
>      10   XX-COMPANY                     'ALPHA ELECTRICAL CO. LTD.'
>      10   XX-DISK                        '8417'
>      10   XX-NO-TERMINALS                 0010
> 
> WORKING-STORAGE
> **********************
> 77        MAX-SUB                         0006
> 77        CUST-STAT                      ALL ZEROES
> 77        REC-NUM                         0001
> 77        UNUSED-VAR                      0001
> 01        BIN                             000000001
> 01        PACKD                           0757.99
>    INDEX  I-ADDR                          +000000001
> 01        TEST-DATA.
>  02       DATA-CUST-NUM-TBL.
>   05      FILLER                         'ALP00000'
>   05      FILLER                         'BET00000'
>   05      FILLER                         'DEL00000'
>   05      FILLER                         'EPS00000'
>   05      FILLER                         'FOR00000'
>   05      FILLER                         'GAM00000'
>  02       DATA-COMPANY-TBL.
>   05      FILLER                         'ALPHA ELECTRICAL CO. LTD.'
>   05      FILLER                         'BETA SHOE MFG. INC.'
>   05      FILLER                         'DELTA LUGGAGE REPAIRS'
>   05      FILLER                         'EPSILON EQUIPMENT SUPPLY'
>   05      FILLER                         'FORTUNE COOKIE COMPANY'
>   05      FILLER                         'GAMMA X-RAY TECHNOLOGY'
>  02       DATA-ADDRESS-2-TBL.
>   05      FILLER                         'ATLANTA'
>   05      FILLER                         'CALGARY'
>   05      FILLER                         'NEW YORK'
>   05      FILLER                         'TORONTO'
>   05      FILLER                         'WASHINGTON'
>   05      FILLER                         'WHITEPLAIN'
>  02       DATA-NO-TERMINALS-TBL.
>   05      FILLER                          010
>   05      FILLER                          013
>   05      FILLER                          075
>   05      FILLER                          010
>   05      FILLER                          090
>   05      FILLER                          254
> 01        TSTREC.                        <NULL> address
> 01        WORK-AREA.
>   05      SUB                             0007
> 
> END OF DUMP - prog
> **********************
> 
846. run_misc.at:9704:  FAILED (run_misc.at:10152)
847. run_misc.at:10157: testing Test dump feature (5) ...
./run_misc.at:10224: SOURCE_DATE_EPOCH=1011960000 \
$COMPILE -fdump=ALL prog.cob
./run_misc.at:10227: export COB_DUMP_FILE=dumpall.txt
$COBCRUN_DIRECT ./prog
./run_misc.at:10285: $SED -e 's/Started by .*prog.exe/Started by .\/prog/g' \
dumpall.txt > dumpall.sed
./run_misc.at:10287: $DIFF reference dumpall.sed
--- /dev/null	2025-05-21 13:01:42.412383655 +0000
+++ /home/appveyor/projects/gnucobol-trunk/tests/testsuite.dir/at-groups/847/stdout	2025-05-21 13:10:30.302160662 +0000
@@ -0,0 +1,4 @@
+19c19
+< 01        BAD-PTR                         0x0000000000000000
+---
+> 01        BAD-PTR                         0x00000000
./run_misc.at:10287: exit code was 1, expected 0
./dumpall.txt:
> 
> Module dump due to module '' not found
> 
>  Last statement of "sub1" was CALL
> 	DMP-TIME1 OF MAIN-1 at prog.cob:59
> 	DMP-TIME OF MAIN-1 at prog.cob:61
> 	MAIN-1 at prog.cob:53
> 	ENTRY sub1 at prog.cob:52
>  Last statement of "prog" was CALL
> 	CALL-SUB-1 at prog.cob:33
> 	MAIN-100 at prog.cob:29
> 	ENTRY prog at prog.cob:24
>  Started by ./prog
> 
> Dump Program-Id sub1 from prog.cob compiled Jan 25 2002 12:00:00
> 
> WORKING-STORAGE
> **********************
> 01        BAD-PTR                         0x00000000
> 
> LINKAGE
> **********************
> 01        X                               000000000
> 01        TSP-RECORD.
>      10   CM-CUST-NUM                    'RON00001'
>      10   CM-COMPANY                     'IBS Canada'
>      10   CM-DISK                        '4TB'
>      10   CM-NO-TERMINALS                 64.699997
> 
> END OF DUMP - sub1
> **********************
> 
> Dump Program-Id prog from prog.cob compiled Jan 25 2002 12:00:00
> 
> WORKING-STORAGE
> **********************
> 77        UNUSED-VAR                      0001
> 01        BIN                             000000000
> 01        PACKD                           0757.99
> 01        TSPFL-RECORD.
>      10   CM-CUST-NUM                    'RON00001'
>      10   CM-COMPANY                     'IBS Canada'
>      10   CM-DISK                        '4TB'
>      10   CM-NO-TERMINALS                 64.699997
> 
> END OF DUMP - prog
> **********************
> 
847. run_misc.at:10157:  FAILED (run_misc.at:10287)
## ------------- ##
## Test results. ##
## ------------- ##
ERROR: All 2 tests were run,
2 failed unexpectedly.
## -------------------------- ##
## testsuite.log was created. ##
## -------------------------- ##
Please send `tests/testsuite.log' and all information you think might help:
   To: <[email protected]>
   Subject: [GnuCOBOL 4.0] testsuite: 846 847 failed
You may investigate any problem if you feel able to do so, in which
case the test suite provides a good starting point.  Its

[ddeclerck]

Test failures on 32bit - so we either need a sed script or a conditional test here

Ah, I knew I had forgotten something.
I'll go for the same sed workaround as one of the tests above.

Maybe this calls for a 32-bit Ubuntu CI (we only test 32-bit on MSYS2 and MSVC).

[GitMensch]

Hm, shouldn't the 32bit MSYS test have failed as well? We may still ignore the failure (but should try to skip/ignore, then error the CI also on Win32)...

Concerning the ubuntu one, this is the different part that we run on appveyor (starting from the already created dev tarball) for the 32bit build on Ubuntu (also testing older GCC)

sudo apt install -y lib32gcc-7-dev libgmp-dev:i386 libdb-dev:i386 libncurses5-dev:i386 libxml2-dev:i386 libicu-dev:i386 libodbc1:i386 odbcinst1debian2:i386 unixodbc-dev:i386

export PKGCONFIG_PATH=/usr/lib32/pkgconfig:/usr/local/lib32/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
./configure CC="gcc-7 -m32" --with-odbc

[ddeclerck]

Hm, shouldn't the 32bit MSYS test have failed as well? We may still ignore the failure (but should try to skip/ignore, then error the CI also on Win32)...

The thing is, test 846 fails anyways on MSYS2 & MSVC because of a segfault (even on 64-bit). 847 is now okay on 32-bit.

Concerning the ubuntu one, this is the different part that we run on appveyor (starting from the already created dev tarball) for the 32bit build on Ubuntu (also testing older GCC)

sudo apt install -y lib32gcc-7-dev libgmp-dev:i386 libdb-dev:i386 libncurses5-dev:i386 libxml2-dev:i386 libicu-dev:i386 libodbc1:i386 odbcinst1debian2:i386 unixodbc-dev:i386

export PKGCONFIG_PATH=/usr/lib32/pkgconfig:/usr/local/lib32/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
./configure CC="gcc-7 -m32" --with-odbc

I'm gonna try to tweak the Ubuntu CI using this info (first on GC3, then on GC4).

[GitMensch]

sure (for GC3 obviously without the odbc/oci part and for configure using --without-db) [I'm back in around 3 hours for a possible review, but if you have a working CI only change, then just commit that...]

[ddeclerck]

Fix merged in SVN @ 5535.