[john-users] opencl on amd (mesa ISC and clc-amd) fails

[Matus UHLAR - fantomas]

Hello,

my today's bleeding jumbo works with my intel GPU, but not with amdgpu:

the "ISO C99 requires whitespace after the macro name" appears at all
formats tried (yet).

apparently there should be space in "-D__MESA__-D__GPU__"
correct?

Device 1: AMD ICELAND (DRM 3.8.0 / 4.9.0-4-amd64, LLVM 3.9.1)
Benchmarking: descrypt-opencl, traditional crypt(3) [DES OpenCL]... Options used: -I /home/uhlar/src/debian/john/bleeding-jumbo/run/kernels -cl-mad-enable -D__MESA__-D__GPU__ -DDEVICE_INFO=10 -DSIZEOF_SIZE_T=8 -DDEV_VER_MAJOR=13 -DDEV_VER_MINOR=0 -D_OPENCL_COMPILER $JOHN/kernels/DES_bs_hash_checking_kernel.cl
Build log: In file included from <built-in>:319:
<command line>:1:17: warning: ISO C99 requires whitespace after the macro name
In file included from input.cl:9:
/home/uhlar/src/debian/john/bleeding-jumbo/run/kernels/opencl_DES_kernel_params.h:17:60: error: token is not a valid binary operator in a preprocessor subexpression
<command line>:1:27: note: expanded from here

Error -11 building kernel $JOHN/kernels/DES_bs_hash_checking_kernel.cl. DEVICE_INFO=10
OpenCL CL_BUILD_PROGRAM_FAILURE (-11) error in common-opencl.c:1109 - clBuildProgram failed.

[Matus UHLAR - fantomas]

Post by Matus UHLAR - fantomas
apparently there should be space in "-D__MESA__-D__GPU__"
/home/uhlar/src/debian/john/bleeding-jumbo/run/kernels/opencl_DES_kernel_params.h:17:60: error: token is not a valid binary operator in a preprocessor subexpression
<command line>:1:27: note: expanded from here
Error -11 building kernel $JOHN/kernels/DES_bs_hash_checking_kernel.cl. DEVICE_INFO=10
OpenCL CL_BUILD_PROGRAM_FAILURE (-11) error in common-opencl.c:1109 - clBuildProgram failed.

I have added in space after the "-D__MESA__ " in common-opencl.c

and the error changed:

Device 1: AMD ICELAND (DRM 3.8.0 / 4.9.0-4-amd64, LLVM 3.9.1)
Benchmarking: mysql-sha1-opencl, MySQL 4.1+ [SHA1 OpenCL]... Options used: -I /home/uhlar/src/debian/john/bleeding-jumbo/run/kernels -cl-mad-enable -D__MESA__ -D__GPU__ -DDEVICE_INFO=10 -DSIZEOF_SIZE_T=8 -DDEV_VER_MAJOR=13 -DDEV_VER_MINOR=0 -D_OPENCL_COMPILER -DTWICE -DOFFSET_TABLE_SIZE=14 -DHASH_TABLE_SIZE=13 -DSHIFT64_OT_SZ=2 -DSHIFT64_HT_SZ=3 -DSHIFT128_OT_SZ=4 -DSHIFT128_HT_SZ=9 -DNUM_LOADED_HASHES=13 -DNUM_INT_KEYS=1 -DSELECT_CMP_STEPS=4 -DBITMAP_SIZE_BITS_LESS_ONE=32767 -DUSE_LOCAL_BITMAPS=1 -DIS_STATIC_GPU_MASK=0 -DCONST_CACHE_SIZE=1925299814 -DLOC_0=-1 -DLOC_1=-1 -DLOC_2=-1 -DLOC_3=-1 $JOHN/kernels/sha1_kernel.cl
Build log: input.cl:139:18: warning: unknown attribute 'max_constant_size' ignored
<unknown>:0:0: in function sha1 void (i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(2)*, i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*): unsupported initializer for address space

Error -11 building kernel $JOHN/kernels/sha1_kernel.cl. DEVICE_INFO=10
OpenCL CL_BUILD_PROGRAM_FAILURE (-11) error in common-opencl.c:1109 - clBuildProgram failed.


just for sure, the -list=opencl-devices output:

Platform #1 name: Clover, version: OpenCL 1.1 Mesa 13.0.6
Device #0 (1) name: AMD ICELAND (DRM 3.8.0 / 4.9.0-4-amd64, LLVM 3.9.1)
Device vendor: AMD
Device type: GPU (LE)
Device version: OpenCL 1.1 Mesa 13.0.6
Driver version: 13.0.6 - Catalyst
Native vector widths: char 16, short 8, int 4, long 2
Preferred vector width: char 16, short 8, int 4, long 2
Global Memory: 3.0 GB
Local Memory: 32.0 KB (Local)
Max memory alloc. size: 1.0 GB
Max clock (MHz): 980
Max Work Group Size: 256
Parallel compute cores: 6
Stream processors: 480 (6 x 80)
Speed index: 470400

[Claudio André]

Device 1: AMD ICELAND (DRM 3.8.0 / 4.9.0-4-amd64, LLVM 3.9.1) >

Benchmarking: mysql-sha1-opencl, MySQL 4.1+ [SHA1 OpenCL]... Options
used: -I /home/uhlar/src/debian/john/bleeding-jumbo/run/kernels
-cl-mad-enable -D__MESA__ -D__GPU__ -DDEVICE_INFO=10 -DSIZEOF_SIZE_T=8
-DDEV_VER_MAJOR=13 -DDEV_VER_MINOR=0 -D_OPENCL_COMPILER -DTWICE
-DOFFSET_TABLE_SIZE=14 -DHASH_TABLE_SIZE=13 -DSHIFT64_OT_SZ=2
-DSHIFT64_HT_SZ=3 -DSHIFT128_OT_SZ=4 -DSHIFT128_HT_SZ=9
-DNUM_LOADED_HASHES=13 -DNUM_INT_KEYS=1 -DSELECT_CMP_STEPS=4
-DBITMAP_SIZE_BITS_LESS_ONE=32767 -DUSE_LOCAL_BITMAPS=1
-DIS_STATIC_GPU_MASK=0 -DCONST_CACHE_SIZE=1925299814 -DLOC_0=-1
-DLOC_1=-1 -DLOC_2=-1 -DLOC_3=-1 $JOHN/kernels /sha1_kernel.cl > Build
log: input.cl:139:18: warning: unknown attribute 'max_constant_size'
ignored > <unknown>:0:0: in function sha1 void (i32 addrspace(1)*, i32
addrspace(1)*, i32 addrspace(1)*, i32 addrspace(2)*, i32 addrspace(1)*,
i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32
addrspace(1)*, i32 addrspace(1)*): unsupported initializer for address space

Short answer, MESA is not supported.

unsupported initializer for address space

- Which address space (local or global)? Why is it creating an
initializer? Not the place to discuss the specification, but, as far as
I know there is no sentence that states "The memory is going to be
initialized". On the contrary.

And finally, notice that this patch https://reviews.llvm.org/D6078 states:
- Sort of depends on clang patch to not emit all LDS objects with
zeroinitializer anymore.

I would say: the initialization is NOT expected or desired.
MESA/llvm/clang has a bug.

Claudio

PS: I know, sorry.

[Matus UHLAR - fantomas]

Post by Matus UHLAR - fantomas

Device 1: AMD ICELAND (DRM 3.8.0 / 4.9.0-4-amd64, LLVM 3.9.1) >

Benchmarking: mysql-sha1-opencl, MySQL 4.1+ [SHA1 OpenCL]... Options
used: -I /home/uhlar/src/debian/john/bleeding-jumbo/run/kernels
-cl-mad-enable -D__MESA__ -D__GPU__ -DDEVICE_INFO=10
-DSIZEOF_SIZE_T=8 -DDEV_VER_MAJOR=13 -DDEV_VER_MINOR=0
-D_OPENCL_COMPILER -DTWICE -DOFFSET_TABLE_SIZE=14
-DHASH_TABLE_SIZE=13 -DSHIFT64_OT_SZ=2 -DSHIFT64_HT_SZ=3
-DSHIFT128_OT_SZ=4 -DSHIFT128_HT_SZ=9 -DNUM_LOADED_HASHES=13
-DNUM_INT_KEYS=1 -DSELECT_CMP_STEPS=4
-DBITMAP_SIZE_BITS_LESS_ONE=32767 -DUSE_LOCAL_BITMAPS=1
-DIS_STATIC_GPU_MASK=0 -DCONST_CACHE_SIZE=1925299814 -DLOC_0=-1
-DLOC_1=-1 -DLOC_2=-1 -DLOC_3=-1 $JOHN/kernels /sha1_kernel.cl >
Build log: input.cl:139:18: warning: unknown attribute
'max_constant_size' ignored > <unknown>:0:0: in function sha1 void
(i32 addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32
addrspace(2)*, i32 addrspace(1)*, i32 addrspace(1)*, i32
addrspace(1)*, i32 addrspace(1)*, i32 addrspace(1)*, i32
addrspace(1)*): unsupported initializer for address space

looking at sha1_kernel.cl:

#if !defined(__OS_X__) && USE_CONST_CACHE && gpu_amd(DEVICE_INFO)
__attribute__((max_constant_size (NUM_INT_KEYS * 4)))
#endif

commenting this out didn't help

Post by Matus UHLAR - fantomas
Short answer, MESA is not supported.

that's unfortunate :-(
I think I've seen mention of this somewhere but it's not said explicitly in
the docs - only recommendation of using vendor-supplied drivers.

Post by Matus UHLAR - fantomas

unsupported initializer for address space

- Which address space (local or global)? Why is it creating an
initializer? Not the place to discuss the specification, but, as far
as I know there is no sentence that states "The memory is going to be
initialized". On the contrary.
- Sort of depends on clang patch to not emit all LDS objects with
zeroinitializer anymore.
I would say: the initialization is NOT expected or desired.
MESA/llvm/clang has a bug.

apparently MESA (I don't have llvm nor clang installed, wouldn't help
apparently).

should I try compile with clang? there's 3.9 and 3.9 available in debian

[magnum]

Post by Claudio André
Short answer, MESA is not supported.

that's unfortunate :-( I think I've seen mention of this somewhere but
it's not said explicitly in
the docs - only recommendation of using vendor-supplied drivers.

Just to clarify what Claudio said, we're (or I'm) aiming for support for
ANY OpenCL compliant gear (as good as possible) but we don't quite
"support" it (we simply can not test every toaster and
firmware/driver/runtime version). OTOH we don't strictly support
*anything* in Jumbo - every single part of it is best-effort. Right now,
due to quality problems with most anything else, NVIDIA (using their
closed-source drivers) works better than most anything else.

magnum

[Matus UHLAR - fantomas]

Post by magnum

Post by Claudio André
Short answer, MESA is not supported.

that's unfortunate :-( I think I've seen mention of this somewhere
but it's not said explicitly in
the docs - only recommendation of using vendor-supplied drivers.

Just to clarify what Claudio said, we're (or I'm) aiming for support
for ANY OpenCL compliant gear (as good as possible) but we don't
quite "support" it (we simply can not test every toaster and
firmware/driver/runtime version). OTOH we don't strictly support
*anything* in Jumbo - every single part of it is best-effort.

of course - that's why I said it's unfortunate. I don't blame anyone, just
seek for information and aim to make it working with my little programming
(especially opencl) knowledge.

Post by magnum
Right
now, due to quality problems with most anything else, NVIDIA (using
their closed-source drivers) works better than most anything else.

I would prefer to have nvidia card over AMD in my company's laptop, however
it was bought by my employer, so I can only can try this one.
on my home PC I'm happy nvidia owner :-)