Test (BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) sometimes fail (on lispworks 8 linux 64bit)

[Yehouda]

Test (BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) sometimes fail, sometimes not.
The output below shows four runs, without doing anything between them, It failed, passed, failed, passed.

It needs to be LispWorks 8 to get it, Lispworks 7 gets an error loading because it doesn't have local-nickanmes.

I also got on AIX (IBM unix) 64bit.

The attached file at the end contains a failing run without :report quite.

CL-USER 13 > (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet)
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 289 x 288 (83232 > 65536 1.2700195)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.919693 1.0912404
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
5: a = 35.424927 1.107029
packed into 285 x 285 (81225 > 65536 1.2393952)
8: a = 34.479294 1.0774779
#<PARACHUTE:QUIET 54056, FAILED results>

CL-USER 14 > (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet)
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 291 x 288 (83808 > 65536 1.2788086)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.923417 1.0913568
packed into 309 x 256 (79104 > 65536 1.2070313)
5: a = 35.466523 1.1083288
8: a = 34.168183 1.0677557
#<PARACHUTE:QUIET 54056, PASSED results>

CL-USER 15 > (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet)
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 289 x 288 (83232 > 65536 1.2700195)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.90236 1.0906987
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
5: a = 35.49434 1.1091981
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44278 1.0763369
#<PARACHUTE:QUIET 54056, FAILED results>

CL-USER 16 > (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet)
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 291 x 288 (83808 > 65536 1.2788086)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.923417 1.0913568
packed into 309 x 256 (79104 > 65536 1.2070313)
5: a = 35.19728 1.099915
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.516083 1.0786276
#<PARACHUTE:QUIET 54056, PASSED results
binpack-noisy.txt

[3b]

Hmm, looks like packing code isn't as deterministic as it should be :/

I think it might be from iterating over hash keys, and it just happened to get consistent ordering on other implementations.

If you want to test https://github.com/3b/binpack/tree/subhole-order that might fix it. Paste a few runs of output with :report 'parachute::quiet either way if you try it though, since I'm curious if it also affects the variation seen in the other output.

[Yehouda]

I don't normally take thing out of github, we test what comes with quicklisp.

I ahve overwrriten the chazelle.lisp file in our version with the fixed version, and now this test doesn't fail anymore, so it is the right fix.

Don't see any variation in the output.

CL-USER 9 > (dotimes (x 10) (print (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet))  (print '-------------------)(terpri) ))
(dotimes (x 10) (print (parachute::test '(BINPACK-TEST/2::BINPACK2 BINPACK-TEST/2::hole1) :report 'parachute::quiet))  (print '-------------------)(terpri) ))
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 
3: a = 32.231934 1.0072479
packed into 256 x 333 (85248 > 65536 1.3007813)
packed into 295 x 288 (84960 > 65536 1.2963867)
packed into 305 x 293 (89365 > 65536 1.3636017)
packed into 292 x 299 (87308 > 65536 1.3322144)
4: a = 34.99344 1.093545
packed into 309 x 256 (79104 > 65536 1.2070313)
packed into 287 x 301 (86387 > 65536 1.318161)
packed into 306 x 304 (93024 > 65536 1.4194336)
5: a = 35.93387 1.1229334
packed into 299 x 272 (81328 > 65536 1.2409668)
8: a = 34.44342 1.0763569

#<PARACHUTE:QUIET 54056, PASSED results> 
------------------- 


```

[3b]

Thanks, that looks much better. Will try to get that released at some point once I have time to test it in actual use.