go tests fail on 32-bit systems #37

rogers0 · 2020-06-25T09:58:18Z

two tests pass on 64-bit systems, but fail on 32-bit systems, such as i386, armhf.

error log:

=== RUN   Test_AsString
    Test_AsString: conversion_test.go:143: 
        	Error Trace:	conversion_test.go:143
        	Error:      	Not equal: 
        	            	expected: "6273346999"
        	            	actual  : "6.273346999e+09"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-6273346999
        	            	+6.273346999e+09
        	Test:       	Test_AsString
--- FAIL: Test_AsString (0.00s)


=== RUN   TestNewCollection
--- FAIL: TestNewCollection (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 21 [running]:
testing.tRunner.func1.1(0x83569a0, 0x863c860)
	/usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0x8ce01e0)
	/usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83569a0, 0x863c860)
	/usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0x8c94f84, 0x1, 0x0, 0xffffffff, 0xf)
	/usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0x8c94f60, 0x8c4ce70)
	/build/1st/golang-github-viant-toolbox-0.32.2/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestNewCollection(0x8ce01e0)
	/build/1st/golang-github-viant-toolbox-0.32.2/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:14 +0x30d
testing.tRunner(0x8ce01e0, 0x83b8410)
	/usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
	/usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL	github.com/viant/toolbox/data	0.032s

Reference:

The text was updated successfully, but these errors were encountered:

adranwit · 2020-06-25T15:54:22Z

I've patched converter test,
However, with the second issue, it looks like using int64 data type cause panic,
If there is a way to detect test architecture I can disable that test ?

var initSize = len(s.fieldNames)
if initSize < len(data) {
	initSize = len(data)
}

line 146# atomic.AddInt64(&s.size, 1)

rogers0 · 2020-06-25T17:01:09Z

for detecting CPU, maybe you can refer:

https://github.com/klauspost/cpuid

rogers0 · 2020-06-26T15:25:56Z

tried latest 0.32.3 on i386, and there're two fails:

# github.com/viant/toolbox_test [github.com/viant/toolbox.test]
src/github.com/viant/toolbox/converter_test.go:409:9: constant 1668069210749 overflows int
src/github.com/viant/toolbox/converter_test.go:521:8: constant 1668069210749 overflows int
FAIL    github.com/viant/toolbox [build failed]
# github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]
src/github.com/viant/toolbox/data/udf/conversion_test.go:142:17: constant 6273346999 overflows int
FAIL    github.com/viant/toolbox/data/udf [build failed]
# github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]
src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int
FAIL    github.com/viant/toolbox/storage [build failed]

another one:

=== RUN   TestCompactedSlice_SortedRange
--- FAIL: TestCompactedSlice_SortedRange (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 8 [running]:
testing.tRunner.func1.1(0x83569a0, 0x863c860)
        /usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0xa4a6320)
        /usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83569a0, 0x863c860)
        /usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0xa466fe4, 0x1, 0x0, 0xfffffff, 0x0)
        /usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0xa466fc0, 0xa413540)
        /build/golang-github-viant-toolbox-0.32.3/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestCompactedSlice_SortedRange(0xa4a6320)
        /build/golang-github-viant-toolbox-0.32.3/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:211 +0x12a6 
testing.tRunner(0xa4a6320, 0x83b8404)
        /usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
        /usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL    github.com/viant/toolbox/data   0.145s
FAIL    github.com/viant/toolbox/data/udf [build failed]

adranwit · 2020-06-29T15:34:30Z

Updated test with skip for non 64 bit arch

rogers0 · 2020-07-02T15:52:59Z

@adranwit Thanks for your update!

For latest 0.33.0, the result is:

# github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]
src/github.com/viant/toolbox/data/udf/conversion_test.go:151:17: constant 2323232323223 overflows int
# github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]
src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int
# github.com/viant/toolbox_test [github.com/viant/toolbox.test]
src/github.com/viant/toolbox/converter_test.go:527:8: constant 1668069210749 overflows int

another one:

=== RUN   TestNewCollection
--- FAIL: TestNewCollection (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804a78c]

goroutine 6 [running]:
testing.tRunner.func1.1(0x83594e0, 0x8642880)
	/usr/lib/go-1.14/src/testing/testing.go:940 +0x27c
testing.tRunner.func1(0x94ac1e0)
	/usr/lib/go-1.14/src/testing/testing.go:943 +0x349
panic(0x83594e0, 0x8642880)
	/usr/lib/go-1.14/src/runtime/panic.go:969 +0x122
runtime/internal/atomic.Xadd64(0x9428fb4, 0x1, 0x0, 0xffffffff, 0xffffff)
	/usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:105 +0xc
github.com/viant/toolbox/data.(*CompactedSlice).Add(0x9428f90, 0x9451e70)
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted.go:146 +0x72
github.com/viant/toolbox/data.TestNewCollection(0x94ac1e0)
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/data/compacted_test.go:17 +0x31b
testing.tRunner(0x94ac1e0, 0x83bbe50)
	/usr/lib/go-1.14/src/testing/testing.go:991 +0xb4
created by testing.(*T).Run
	/usr/lib/go-1.14/src/testing/testing.go:1042 +0x2ad
FAIL	github.com/viant/toolbox/data	0.036s

adranwit · 2020-07-07T15:22:51Z

Looks like 64 detection, I've added to the test does not work,
if cpuid.CPU.Family < 64 {
return
}

could you please print me the following on the machine the test failed ?

fmt.Println("Name:", cpuid.CPU.BrandName)
fmt.Println("PhysicalCores:", cpuid.CPU.PhysicalCores)
fmt.Println("ThreadsPerCore:", cpuid.CPU.ThreadsPerCore)
fmt.Println("LogicalCores:", cpuid.CPU.LogicalCores)
fmt.Println("Family", cpuid.CPU.Family, "Model:", cpuid.CPU.Model)
fmt.Println("Features:", cpuid.CPU.Features)
fmt.Println("Cacheline bytes:", cpuid.CPU.CacheLine)
fmt.Println("L1 Data Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L1 Instruction Cache:", cpuid.CPU.Cache.L1D, "bytes")
fmt.Println("L2 Cache:", cpuid.CPU.Cache.L2, "bytes")
fmt.Println("L3 Cache:", cpuid.CPU.Cache.L3, "bytes")

rogers0 · 2020-07-07T15:40:12Z

For debian, I used the patch:
https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0001-Skip-some-tests-fail-at-git-pbuilder-environment.patch

This most important part is:

-	if cpuid.CPU.CacheLine < 64 {
-		return
+	if (32 << uintptr(^uintptr(0)>>63)) < 64 {
+		t.Skip()

rogers0 · 2020-07-07T16:00:33Z

For debian, I used the patch:
https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0001-Skip-some-tests-fail-at-git-pbuilder-environment.patch

Let me clarify. I do not mean you should merge this patch to upstream repository.
I just show you how I workaround current errors from reproducible builds.
Thanks for your working!

adranwit · 2020-07-08T15:35:46Z

Thanks for the suggestion, updated tests

adranwit · 2020-07-15T18:37:24Z

Have you run test with the latest code ?

rogers0 · 2020-07-19T17:03:15Z

Sorry for the late reply.
The test code still build fails under 32-bit systems, so I have to disable it.

https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/rules#L17

But I still think some code need to be updated:

https://salsa.debian.org/go-team/packages/golang-github-viant-toolbox/-/blob/debian/sid/debian/patches/0002-Skip-some-tests-fail-at-git-pbuilder-environment.patch

I'll send you a PR later.

adranwit · 2020-07-27T18:32:33Z

Removed 32 bit test

rogers0 · 2020-08-02T12:46:42Z

Except PR #39
there're a few other issues.
(since I set ignore all 32-bit go test errors, so currently reproducible builds all "passed")

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/golang-github-viant-toolbox.html

All current log based on latest, + PR #39 patch:

test fails on i386 (x86 32-bit)

=== RUN   TestConverter_AsInt
    TestConverter_AsInt: converter_test.go:702: 
        	Error Trace:	converter_test.go:702
        	Error:      	Should be true
        	Test:       	TestConverter_AsInt
--- FAIL: TestConverter_AsInt (0.00s)

https://tests.reproducible-builds.org/debian/rbuild/unstable/i386/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

panic: runtime error on i386 (x86 32-bit) and armhf (arm 32-bit)

=== RUN   TestLogger
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x804ab9b]

goroutine 165 [running]:
runtime/internal/atomic.Load64(0xa86b2ec, 0xbfc1ad5d, 0xd905ff2)
	/usr/lib/go-1.14/src/runtime/internal/atomic/asm_386.s:194 +0xb
github.com/viant/toolbox.(*LogStream).isFrequencyFlushNeeded(0xa86b2c0, 0x0)
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:138 +0x69
github.com/viant/toolbox.(*LogStream).manageWritesInBatch(0xa86b2c0)
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:166 +0x23b
github.com/viant/toolbox.(*FileLogger).NewLogStream.func1(0xa86b2c0)
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:219 +0x21
created by github.com/viant/toolbox.(*FileLogger).NewLogStream
	/build/1st/golang-github-viant-toolbox-0.33.0/obj-i686-linux-gnu/src/github.com/viant/toolbox/file_logger.go:218 +0xfe
FAIL	github.com/viant/toolbox	0.239s

https://tests.reproducible-builds.org/debian/rbuild/unstable/armhf/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

build error on 32-bit systems:

# github.com/viant/toolbox/data/udf [github.com/viant/toolbox/data/udf.test]
src/github.com/viant/toolbox/data/udf/conversion_test.go:142:17: constant 2323232323223 overflows int
# github.com/viant/toolbox/storage_test [github.com/viant/toolbox/storage.test]
src/github.com/viant/toolbox/storage/file_info_test.go:13:3: constant 2147484141 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:14:3: constant 2147484159 overflows int
src/github.com/viant/toolbox/storage/file_info_test.go:15:3: constant 2147484128 overflows int

https://tests.reproducible-builds.org/debian/rbuild/unstable/i386/golang-github-viant-toolbox_0.33.0-11.rbuild.log.gz

Thank you!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

go tests fail on 32-bit systems #37

go tests fail on 32-bit systems #37

rogers0 commented Jun 25, 2020

adranwit commented Jun 25, 2020

rogers0 commented Jun 25, 2020

rogers0 commented Jun 26, 2020 •

edited

Loading

adranwit commented Jun 29, 2020

rogers0 commented Jul 2, 2020

adranwit commented Jul 7, 2020

rogers0 commented Jul 7, 2020

rogers0 commented Jul 7, 2020

adranwit commented Jul 8, 2020

adranwit commented Jul 15, 2020

rogers0 commented Jul 19, 2020

adranwit commented Jul 27, 2020

rogers0 commented Aug 2, 2020

go tests fail on 32-bit systems #37

go tests fail on 32-bit systems #37

Comments

rogers0 commented Jun 25, 2020

adranwit commented Jun 25, 2020

rogers0 commented Jun 25, 2020

rogers0 commented Jun 26, 2020 • edited Loading

adranwit commented Jun 29, 2020

rogers0 commented Jul 2, 2020

adranwit commented Jul 7, 2020

rogers0 commented Jul 7, 2020

rogers0 commented Jul 7, 2020

adranwit commented Jul 8, 2020

adranwit commented Jul 15, 2020

rogers0 commented Jul 19, 2020

adranwit commented Jul 27, 2020

rogers0 commented Aug 2, 2020

rogers0 commented Jun 26, 2020 •

edited

Loading