4

Result of below investigation is: Recent Node.js is not portable to AMD Geode (or other non-SSE x86) Processors !!!

I dived deeper into the code and got stuck in ia32-assembler implementation, that deeply integrates SSE/SSE2 instructions into their code (macros, macros, macros,...). The main consequence is, that you can not run a recent version of node.js on AMD geode processors due to the lack of newer instuction set extensions. The fallback to 387 arithmetics only works for the node.js code, but not for the javascript V8 compiler implementation that it depends on. Adjusting V8 to support non-SSE x86 processors is a pain and a lot of effort.

If someone produces proof of the contrary, I would be really happy to hear about ;-)

Investigation History

I have a running ALIX.2D13 (https://www.pcengines.ch), which has an AMD Geode LX as the main processor. It runs voyage linux, a debian jessi based distribution for resource restricted embedded devices.

     root@voyage:~# cat /proc/cpuinfo 
     processor       : 0
     vendor_id       : AuthenticAMD
     cpu family      : 5
     model           : 10
     model name      : Geode(TM) Integrated Processor by AMD PCS
     stepping        : 2
     cpu MHz         : 498.004
     cache size      : 128 KB
     physical id     : 0
     siblings        : 1
     core id         : 0
     cpu cores       : 1
     apicid          : 0
     initial apicid  : 0
     fdiv_bug        : no
     f00f_bug        : no
     coma_bug        : no
     fpu             : yes
     fpu_exception   : yes
     cpuid level     : 1
     wp              : yes
     flags           : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow 3dnowprefetch vmmcall
     bugs            : sysret_ss_attrs
     bogomips        : 996.00
     clflush size    : 32
     cache_alignment : 32
     address sizes   : 32 bits physical, 32 bits virtual

When I install nodejs 8.x following the instructions on https://nodejs.org/en/download/package-manager/, I get some "invalid machine instruction" (not sure if correct, but translated from german error output). This also happens, when I download the binary for 32-bit x86 and also when I compile it manually.

After the answers below, I changed the compiler flags in deps/v8/gypfiles/toolchain.gypi by removing -msse2 and adding -march=geode -mtune=geode. And now I get the same error but with a stack trace:

root@voyage:~/GIT/node# ./node


#
# Fatal error in ../deps/v8/src/ia32/assembler-ia32.cc, line 109
# Check failed: cpu.has_sse2().
#

==== C stack trace ===============================

    ./node(v8::base::debug::StackTrace::StackTrace()+0x12) [0x908df36]
    ./node() [0x8f2b0c3]
    ./node(V8_Fatal+0x58) [0x908b559]
    ./node(v8::internal::CpuFeatures::ProbeImpl(bool)+0x19a) [0x8de6d08]
    ./node(v8::internal::V8::InitializeOncePerProcessImpl()+0x96) [0x8d8daf0]
    ./node(v8::base::CallOnceImpl(int*, void (*)(void*), void*)+0x35) [0x908bdf5]
    ./node(v8::internal::V8::Initialize()+0x21) [0x8d8db6d]
    ./node(v8::V8::Initialize()+0xb) [0x86700a1]
    ./node(node::Start(int, char**)+0xd3) [0x8e89f27]
    ./node(main+0x67) [0x846845c]
    /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0xb74fc723]
    ./node() [0x846a09c]
Ungültiger Maschinenbefehl
root@voyage:~/GIT/node#

If you now look into this file, you will find the following

... [line 107-110]
void CpuFeatures::ProbeImpl(bool cross_compile) {
base::CPU cpu;
CHECK(cpu.has_sse2());  // SSE2 support is mandatory.
CHECK(cpu.has_cmov());  // CMOV support is mandatory.
...

I commented the line but still the "Ungültiger Maschinenbefehl" (Invalid machine instruction).

This is what gdb ./node shows (executed run):

root@voyage:~/GIT/node# gdb ./node
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
[...]
This GDB was configured as "i586-linux-gnu".
[...]
Reading symbols from ./node...done.
(gdb) run
Starting program: /root/GIT/node/node 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
[New Thread 0xb7ce2b40 (LWP 29876)]
[New Thread 0xb74e2b40 (LWP 29877)]
[New Thread 0xb6ce2b40 (LWP 29878)]
[New Thread 0xb64e2b40 (LWP 29879)]

Program received signal SIGILL, Illegal instruction.
0x287a23c0 in ?? ()
(gdb)

I think, it is necessary to compile with debug symbols...

make clean
make CFLAGS="-g"

No chance to resolve all SSE/SSE2-Problems... Giving up! See my topmost section

themole
  • 313
  • 2
  • 11
  • If V8's code thinks SSE2 support is mandatory, the 387 fallback code maybe isn't maintained / correct anymore. I'd be careful, and maybe ask on a libV8 mailing list whether it was safe to use. They might want to know something about your use-case, but Javascript numbers are floating-point by default, and being able to use integer code is only possible as an optimization when the JIT-compiler can prove it gives the same results. – Peter Cordes Jul 20 '17 at 16:34
  • @Peter: You are totally right. I gave up on this. I'll buy an OrangePi or RaspberryPi 3 to get some little nodejs server for home automation. Sad to say that this renders the bunch of ALIX boards in my garage useless :-( – themole Jul 21 '17 at 07:42
  • If home automation is all you were doing, maybe consider getting V8 to interpret only, and disable its JIT? That might still be fast enough for your needs. Does it have a mode that doesn't take advantage of running on ia32 or x86-64? Because then it would hopefully just be a pure C program that gcc could compile for your Geode. – Peter Cordes Jul 21 '17 at 07:53
  • @Peter: C is not the right language to implement that part of my software. But I can live with using a respberry for that perfectly. – themole Jul 21 '17 at 12:11
  • What I meant was that nodes.js+V8 itself is just a C++ program, if you can get it to run in a mode where it only interprets without JITing. It might have such a mode intended for use on non-x86 CPU architectures where V8 doesn't have a JIT engine. But if you can get it to do that on x86, you're all set except for performance. – Peter Cordes Jul 21 '17 at 16:56
  • As I read about it, it sounds like V8 has no interpreter. It is always compiling the JavaScript code. – themole Jul 21 '17 at 17:51
  • I had been assuming that V8 used a similar strategy to the HotSpot JVM: interpret while profiling or when hitting an optimization-blocker (I know some JS code "deoptimizes"), then JIT optimized code once it starts up. But apparently they don't do that... yet. An interpreter is a planned new feature in V8: https://github.com/v8/v8/wiki/Interpreter. It makes sense: Java bytecode is already compiled into an interpreter-friendly format, but JS isn't. V8's plan is to compile to bytecode internally before interpreting/JITing, instead of just making un-optimized JIT machine code. – Peter Cordes Jul 21 '17 at 19:05
  • I updated my answer, including a suggestion that you could maybe run `node.js` on top of `qemu` to emulate an x86 with SSE2 on top of an x86 without. It should be near native speed for integer code. – Peter Cordes Jul 21 '17 at 19:23

1 Answers1

2

Conclusion: node.js + V8 normally requires SSE2 when running on x86.

On the V8 ports page: x87 (not officially supported)

Contact/CC the x87 team in the CL if needed. Use the mailing list v8-x87-ports.at.googlegroups.com for that purpose.

Javascript generally requires floating point (every numeric variable is floating point, and using integer math is only an optimization), so it's probably hard to avoid having V8 actually emit FP math instructions.

V8 is currently designed to always JIT, not interpret. It starts off / falls-back to JITing un-optimized machine code when it's still profiling, or when it hits something that makes it "de-optimize".

There is an effort to add an interpreter to V8, but it might not help because the interpreter itself will be written using the TurboFan JIT backend. It's not intended to make V8 portable to architectures it doesn't currently know how to JIT for.

Crazy idea: run node.js on top of a software emulation layer (like Intel's SDE or maybe qemu-user) that could emulate x86 with SSE/SSE2 on an x86 CPU supporting only x87. They use dynamic translation, so would probably run at near-native speed for code that didn't use any SSE instructions.

This may be crazy because node.js + V8 probably some virtual-memory tricks that might confuse an emulation layer. I'd guess that qemu should be robust enough, though.

Original answer left below as a generic guide to investigating this kind of issue for other programs. (tip: grep the Makefiles and so on for -msse or -msse2, or check compiler command lines for that with pgrep -a gcc while it's building).

Your cpuinfo says it has CMOV, which is a 686 (ppro / p6) feature. This says that Geode supports i686. What's missing compared to a "normal" CPU is SSE2, which is enabled by default for -m32 (32-bit mode) in some recent compiler versions.

Anyway, what you should do is compile with -march=geode -O3, so gcc or clang will use everything your CPU supports, but no more.

-O3 -msse2 -march=geode would tell gcc that it can use everything Geode supports as well as SSE2, so you need to remove any -msse and -msse2 options, or add -mno-sse after them. In node.js, deps/v8/gypfiles/toolchain.gypi was setting -msse2.

Using -march=geode implies -mtune=geode, which affects code-gen choices that don't involve using new instructions, so with luck your binary will run faster than if you'd simply used -mno-sse to control instruction-set stuff without overriding -mtune=generic. (If you're building on the geode, you could use -march=native, which should be identical to using -march=geode.)

The other possibility is the problem instructions are in Javascript functions that were JIT-compiled.

node.js uses V8. I did a quick google search, but didn't find anything about telling V8 to not assume SSE/SSE2. If it doesn't have a fall-back code-gen strategy (x87 instructions) for floating point, then you might have to disable JIT altogether and make it run in interpreter mode. (Which is slower, so that may be a problem.)

But hopefully V8 is well-behaved, and checks what instruction sets are supported before JITing.

You should check by running gdb /usr/bin/node, and see where it faults. Type run my_program.js on the GDB command line to start the program. (You can't pass args to node.js when you first start gdb. You have to specify args from inside gdb when you run.)

If the address of the instruction that raised SIGILL is in a region of memory that's mapped to a file (look in /proc/pid/maps if gdb doesn't tell you), that tells you which ahead-of-time compiled executable or library is responsible. Recompile it with -march=geode.

If it's in anonymous memory, it's most likely JIT-compiler output.

GDB will print the instruction address when it stops when the program receives SIGILL. You can also print $ip to see the current value of EIP (the 32-bit mode instruction pointer).

Community
  • 1
  • 1
Peter Cordes
  • 245,674
  • 35
  • 423
  • 606
  • Since this is not the usual autotools configure script, can you tell how or where to pass the `-march` compiler flag? I would like to try that first, since it has the highest chance to work in my opinion. – themole Jul 16 '17 at 18:02
  • @themole: no clue, I haven't built node.js from source. I just know about CPUs, and instruction sets, and compiler options. I've only used node.js from binary packages. But I did a quick google and found https://stackoverflow.com/questions/10391358/node-js-build-configure-options-explanation which says to just look in `./configure` for the options it supports. But note that it might be libV8 that's the problem, not node.js *itself*. So you might want to start with gdb to see which binary has the illegal instruction. GDB should print this for you without you having to search `/proc/x/maps` – Peter Cordes Jul 16 '17 at 20:23
  • @themole: I was curious, so I searched some more: https://github.com/nodejs/node-v0.x-archive/issues/5195#issuecomment-15774981 says that after you run `./configure`, build with `make CFLAGS="-march=geode -O3" CXXFLAGS="-march=geode -O3"` – Peter Cordes Jul 16 '17 at 20:26
  • 1
    I found option setting `-msse2` inside `deps/v8/gypfiles/toolchain.gypi`, depending on the architecture `ia32`. Beside this, I added the `-march=geode` to the `ia32` architecture switch in `common.gypi` I believe this is the spot, where to turn the screws... Since it takes a few hours to build on my little ALIX board, I'll come back when it is finished and report if it worked. – themole Jul 16 '17 at 20:58
  • @themole: Nice find. `-march=geode -msse2` enables Geode *and* SSE2. **You need to remove `-msse2`**! You could even add `-march=geode -mno-sse` to disable it if any earlier options enabled it. So you're not cross-compiling? That would be faster, once you get a build environment set up. – Peter Cordes Jul 16 '17 at 21:05
  • After compiling it with the changed options (removed `-msse2` and added `-march=geode`) running `./node` gives another error and a stack trace. See edit in my original post. – themole Jul 20 '17 at 15:34