Trying to compile x86 with AVX assembly file

Question

I am trying to compile the following assembly code with command:

nasm -f elf AvxScalarFloatingPointArithmetic_.asm

Assembly code:

    .model flat,c
        .const
AbsMask qword 7fffffffffffffffh, 7fffffffffffffffh
        .code

; extern "C" void AvxSfpArithmetic_(double a, double b, double results[8]);
;
; Description:  The following function demonstrates how to use basic
;               scalar DPFP arithmetic instructions.
;
; Requires:     AVX

AvxSfpArithmetic_ proc
        push ebp
        mov ebp,esp

; Load argument values
        mov eax,[ebp+24]                    ;eax = ptr to results array
        vmovsd xmm0,real8 ptr [ebp+8]       ;xmm0 = a
        vmovsd xmm1,real8 ptr [ebp+16]      ;xmm1 = b

; Perform basic arithmetic using AVX scalar DPFP instructions
        vaddsd xmm2,xmm0,xmm1               ;xmm2 = a + b
        vsubsd xmm3,xmm0,xmm1               ;xmm3 = a - b
        vmulsd xmm4,xmm0,xmm1               ;xmm4 = a * b
        vdivsd xmm5,xmm0,xmm1               ;xmm5 = a / b
        vmovsd real8 ptr [eax+0],xmm2       ;save a + b
        vmovsd real8 ptr [eax+8],xmm3       ;save a - b
        vmovsd real8 ptr [eax+16],xmm4      ;save a * b
        vmovsd real8 ptr [eax+24],xmm5      ;save a / b

; Compute min(a, b), max(a, b), sqrt(a) and fabs(b)
        vminsd xmm2,xmm0,xmm1               ;xmm2 = min(a, b)
        vmaxsd xmm3,xmm0,xmm1               ;xmm3 = max(a, b)
        vsqrtsd xmm4,xmm0,xmm0              ;xmm4 = sqrt(a)
        vandpd xmm5,xmm1,xmmword ptr [AbsMask]  ;xmm5 = fabs(b)
        vmovsd real8 ptr [eax+32],xmm2      ;save min(a, b)
        vmovsd real8 ptr [eax+40],xmm3      ;save max(a, b)
        vmovsd real8 ptr [eax+48],xmm4      ;save sqrt(a)
        vmovsd real8 ptr [eax+56],xmm5      ;save trunc(sqrt(a))

        pop ebp
        ret
AvxSfpArithmetic_ endp
        end

Unfortunatelly there are some errors:

AvxScalarFloatingPointArithmetic_.asm:1: error: attempt to define a local label before any non-local labels
AvxScalarFloatingPointArithmetic_.asm: error: parser: instruction expected
AvxScalarFloatingPointArithmetic_.asm:2: error: attempt to define a local label before any non-local labels
AvxScalarFloatingPointArithmetic_.asm:3: error: parser: instruction expected
AvxScalarFloatingPointArithmetic_.asm:13: error: parser: instruction expected

How should I compile this file? The code should be correct. It is from the book: "Modern X86 Assembly Language Programming: 32-bit, 64-bit, SSE, and AVX".

That's not `nasm` syntax. Looks more like `masm`. Use that, or else you will need to adjust the syntax. If it's from a book I assume they told you what assembler to use. — Jester, May 23 '16 at 21:41
From the book's blurb on Amazon: *"The primary audience for the book is Windows software developers since the sample code will be created using Visual C++ and Microsoft Macro Assembler."* — Paul R, May 23 '16 at 21:46
See [this article](http://left404.com/2011/01/04/converting-x86-assembly-from-masm-to-nasm-3/) for tips on converting MASM source to NASM. — Paul R, May 23 '16 at 21:49

score 3 · Accepted Answer · edited May 23 '17 at 11:52

It looks like MASM syntax, not NASM syntax. Porting to NASM is simple enough if you know NASM syntax, but teaching you that beyond the scope of an SO answer. See the manual, and the x86 tag wiki.

JWasm is a portable MASM-syntax assembler, but be careful of ABI differences. This can save the trouble of porting to NASM syntax, but not of porting Windows API / library calls to Linux system calls / library calls.

If you do all your I/O and other system calls from C, and do data processing in your asm functions, you will be fine (except for calling-convention differences between the Windows and SystemV ABIs).

However, if you have asm functions that use Windows API/library functions directly, it's more complicated. Those APIs don't exist on Linux (unless we're talking about C99 standard library functions like printf).

You could in theory run your Windows code under wine, but you'll have an easier time debugging normal Linux executables, since strace and gdb will be usable directly.

In this case, you could do something like:

default rel

section .rodata
AbsMask dq 7fffffffffffffffh, 7fffffffffffffffh

section .text
; extern "C" void AvxSfpArithmetic_(double a, double b, double results[8]);
global AvxSfpArithmetic
AvxSfpArithmetic:
       push ebp
       mov ebp,esp       ; you don't need to waste instructions on this stack frame crap, as you will soon learn.

; Load argument values
        ;mov eax,[esp+20]                    ;eax = ptr to results array
        vmovsd xmm0,  [ebp+8]       ;xmm0 = a
        vmovsd xmm1,  [ebp+16]      ;xmm1 = b
        ...

I just removed real8 ptr since the instruction implies the operand-size just fine. qword [ebp+8] would work.

If you were going to just load that AND-mask into a register (before a loop) instead of using it directly from memory, you should instead consider generating it on the fly with pcmpeqw xmm7,xmm7 / psrlq xmm7, 1.

in 64bit code, the SystemV ABI (used on Linux) differs from Windows, so if the 64bit examples in your book use the Windows calling convention, you'll have to deal with that. You could use __attribute__((ms_abi)) on the C prototype.

The 32bit SysV ABI uses a calling that's basically compatible with Window's, with all args on the stack. IDK if it differs at all, e.g. for struct returns.

There **are** Linux assemblers that do understand MASM (compatible) syntax. Try using [JWasm](https://github.com/JWasm/JWasm). — zx485, May 24 '16 at 16:04

Trying to compile x86 with AVX assembly file

1 Answers1