Crash or stuck in FORM-internal loop

[nikolai-husung]

The original problem was that I ran into a FORM error message stating:

Illegal code in Norm
Illegal code in Norm
Called from TestSub
Program terminating at prepareDimReg Line 103 --> 
Terminate called from proces.c:2156 (TestSub)
Backtrace:
# 0: TerminateImpl at startup.c:1874:10
# 1: TestSub at proces.c:2156:2
# 2: Generator at proces.c:3286:5
# 3: Generator at proces.c:4096:18
# 4: Generator at proces.c:4096:18
# 5: Generator at proces.c:4264:8
# 6: Processor at proces.c:407:11
# 7: DoExecute at execute.c:859:67
# 8: PreProcessor at pre.c:1048:17
# 9: main at startup.c:1750:2

While trying to build a minimal example to reproduce the problem I ran into two more variants all related with the replacement in lines 85-88 of the attached FORM script:

(1) I end up in a FORM-internal infinite(?) loop when running the script with -D i=1 or -D i=2 input data.
(2) If I remove in line 88 the overall ft1-factor the script still gets stuck with -D i=1 but crashes with -D i=2 without stating a reason:

Program terminating at minError.frm Line 85 --> 
Terminate called from startup.c:1611 (onErrSig)
Backtrace:
# 0: TerminateImpl at startup.c:1874:10
# 1: ?? at startup.c:1611:2
  (inlined by) onErrSig at startup.c:1597:13
# 2: __restore_rt at ??:0
# 3: TestSub at proces.c:705:8
# 4: TestSub at proces.c:1674:13
# 5: Generator at proces.c:3286:5
# 6: Generator at proces.c:4096:18
# 7: Generator at proces.c:4096:18
# 8: Generator at proces.c:4264:8
# 9: Processor at proces.c:407:11
#10: DoExecute at execute.c:859:67
#11: PreProcessor at pre.c:1048:17
#12: main at startup.c:1750:2

(3) Specifying further -D workaround runs my current workaround for the problem in lines 75-82 for any of the choices.

For now I only share the "minimal" example. I can try to build a more compact variant of the original code but it is right now rather lengthy. Let me know if you need any more details and thank you for any help with this.

input1.txt

input2.txt

minError.frm.txt

[jodavies]

Thanks for the report! This looks fine in 4.2.1 (assuming the result is correct) but bad in 4.3.0. Also there is a valgrind error. I'll take a look.

[jodavies]

The first bad commit is b2def9d , which is part of the fix for #307 .

[tueda]

Simplifying the program led me to the following example, which generates infinite output (be cautious when executing it):

CF f,g;
S x,y;
L F1 = f(g(x))*replace_(x,0);    * OK: f(g(0))
L F2 = f(g(-x))*replace_(x,1);   * OK: f(g(-1))
L F3 = f(g(-x))*replace_(x,0);   * BAD (infinite output): 0/0*f(g(0))*g(0/00/00/0 + ...
L F4 = f(g(x+y))*replace_(x,0);  * BAD: 0/0*f(g(y))
P;
.end

[tueda]

Related to #367?

[jodavies]

Right, I thought this code was familiar... it was again also edited in 1d4b775 . I'll try to work out the logic for the term-size updates. The relevance of "-x" means it could be a fast-notation problem? f(x) and f(0) are the same size, f(-x) and f(0) are not? Though, it works when replacing x with 1 instead.

[jodavies]

Commenting out 24f1772 also lets this run though. Maybe the question is, "is it correct for Normalize to set *t to 0 here"? In the replace_(x,1) case, Normalize transforms 8 16 4 1 1 1 1 -3 into 4 1 1 -3. With replace_(x,0) we start with 8 16 4 0 1 1 1 -3 and get 0 back.

[jodavies]

I think I have it:

form/sources/proces.c

Lines 1618 to 1657 in a405343

	i = *t;
	AN.subsubveto = 1;
	/*
	AN.subsubveto repairs a bug that became apparent
	in an example by York Schroeder:
	f(k1.k1)*replace_(k1,2*k2)
	Is it possible to repair the counting of the various
	length indicators? (JV 1-jun-2010)
	*/
	if ( ( retvalue = TestSub(BHEAD t,level) ) != 0 ) {
	/*
	Possible size changes:
	Note defs at 471,467,460,400,425,328
	*/
	redosize:
	if ( i > *t ) {
	/*
	i -= *t;
	*t2 -= i;
	t1[1] -= i;
	t += *t;
	r = t + i;
	m = term + *term;
	while ( r < m ) *t++ = *r++;
	*term -= i;
	*/
	i -= *t;
	t += *t;
	r = t + i;
	m = AN.EndNest;
	while ( r < m ) *t++ = *r++;
	t = AT.NestPoin[-1].argsize + ARGHEAD;
	n = AT.Nest;
	while ( n < AT.NestPoin ) {
	*(n->argsize) -= i;
	*(n->funsize) -= i;
	*(n->termsize) -= i;
	n++;
	}
	AN.EndNest -= i;

Here, the change in the sizes is applied to the whole stack of nesting pointers. The problem is that TestSub is called recursively on L1627, so internally modifies the NestPoin stack. After returning, we still hold the original size i (from L1618) before before the call, so update the sizes of the remaining NestPoin stack for a second time.

The fix is probably to remove the loop over the stack, and only adjust the lengths in the final entry. Then the recursive calls of TestSub deal with the rest. I'll test some more and prepare a PR if things work...

[ArashPartow]

My hats off to you guys for being able to understand this codebase and be productive in it. 👍