How do I play Opus encoded audio in Java?

Question

When playing back the decoded audio, I've managed to produce a variety of sounds from gurgling to screeching to demonic chants. The closest of which sounds similar to being played in fast-forward and playback only lasts about 15 seconds. I've tried with a large combination of parameters for the decoding and AudioSystem API methods, nothing seems to be working.

So, what is causing this audio distortion?

Opusinfo for this file shows the following:

Processing file "test.opus"...

New logical stream (#1, serial: 00002c88): type opus
Encoded with libopus 1.1
User comments section follows...
     ENCODER=opusenc from opus-tools 0.1.9
Opus stream 1:
    Pre-skip: 356
    Playback gain: 0 dB
    Channels: 1
    Original sample rate: 44100Hz
    Packet duration:   20.0ms (max),   20.0ms (avg),   20.0ms (min)
    Page duration:   1000.0ms (max),  996.8ms (avg),  200.0ms (min)
    Total data length: 1930655 bytes (overhead: 1.04%)
    Playback length: 4m:09.173s
    Average bitrate: 61.99 kb/s, w/o overhead: 61.34 kb/s
Logical stream 1 ended

This file plays back correctly using VLC.

To decode the file I'm attempting to use the following libraries:

• VorbisJava (https://github.com/Gagravarr/VorbisJava/) - To pull the frames from the OGG container

• LibJitsi (https://jitsi.org/Projects/LibJitsi) - Which has a JNI wrapper for Opus, used to decode the opus frames

SSCCE below

package me.justinb.mediapad.audio;

import org.gagravarr.ogg.OggFile;
import org.gagravarr.ogg.OggPacket;
import org.jitsi.impl.neomedia.codec.audio.opus.Opus;
import javax.sound.sampled.*;
import java.io.*;
import java.nio.ByteBuffer;

public class OpusAudioPlayer {
    private static int BUFFER_SIZE = 1024 * 1024;
    private static int INPUT_BITRATE = 48000;
    private static int OUTPUT_BITRATE = 44100;
    private OggFile oggFile;
    private long opusState;
    private ByteBuffer decodeBuffer = ByteBuffer.allocate(BUFFER_SIZE); 
    private AudioFormat audioFormat = new AudioFormat(OUTPUT_BITRATE, 16, 1, true, false);

    public static void main(String[] args) {
        try {
            OpusAudioPlayer opusAudioPlayer = new OpusAudioPlayer(new File("test.opus"));
            opusAudioPlayer.play();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public OpusAudioPlayer(File audioFile) throws IOException {
        oggFile = new OggFile(new FileInputStream(audioFile));
        opusState = Opus.decoder_create(INPUT_BITRATE, 1);
        System.out.println("Audio format: " + audioFormat);
    }

    private byte[] decode(byte[] packetData) {
        int frameSize = Opus.decoder_get_nb_samples(opusState, packetData, 0, packetData.length);
        int decodedSamples = Opus.decode(opusState, packetData, 0, packetData.length, decodeBuffer.array(), 0, frameSize, 0);
        if (decodedSamples < 0) {
            System.out.println("Decode error: " + decodedSamples);
            decodeBuffer.clear();
            return null;
        }
        decodeBuffer.position(decodedSamples * 2); // 2 bytes per sample
        decodeBuffer.flip();

        byte[] decodedData = new byte[decodeBuffer.remaining()];
        decodeBuffer.get(decodedData);
        decodeBuffer.flip();
        System.out.println(String.format("Encoded frame size: %d bytes", packetData.length));
        System.out.println(String.format("Decoded frame size: %d bytes", decodedData.length));
        System.out.println(String.format("Decoded %d samples", decodedSamples));
        return decodedData;
    }

    public void play() {
        int totalDecodedBytes = 0;
        try {
            SourceDataLine speaker = AudioSystem.getSourceDataLine(audioFormat);
            OggPacket nextPacket = oggFile.getPacketReader().getNextPacket();
            // Move to beginning of stream
            while ( !nextPacket.isBeginningOfStream()) {
                nextPacket = oggFile.getPacketReader().getNextPacket();
            }
            speaker.open();
            speaker.start();
            while(nextPacket != null) {
                // Decode each packet
                byte[] decodedData = decode(nextPacket.getData());
                if(decodedData != null) {
                    // Write packet to SourceDataLine
                    speaker.write(decodedData, 0, decodedData.length);
                    totalDecodedBytes += decodedData.length;
                }
                nextPacket = oggFile.getPacketReader().getNextPacket();
            }
            speaker.drain();
            speaker.close();
            System.out.println(String.format("Decoded to %d bytes", totalDecodedBytes));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Answer 1

My particular issue seemed to be caused by a bug in VorbisJava. I'm now using J-Ogg which is handling the container parsing without any problems. I'm certain someone will find this useful.

This is the final code which shows how to play Opus encoded audio in Java:

package me.justinb.mediapad.audio;

import de.jarnbjo.ogg.FileStream;
import de.jarnbjo.ogg.LogicalOggStream;
import org.jitsi.impl.neomedia.codec.audio.opus.Opus;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.SourceDataLine;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.util.Collection;

public class OpusAudioPlayer {
    private static int BUFFER_SIZE = 1024 * 1024;
    private static int INPUT_BITRATE = 48000;
    private static int OUTPUT_BITRATE = 48000;

    private FileStream oggFile;
    private long opusState;

    private ByteBuffer decodeBuffer = ByteBuffer.allocate(BUFFER_SIZE);

    private AudioFormat audioFormat = new AudioFormat(OUTPUT_BITRATE, 16, 1, true, false);

    public static void main(String[] args) {
        try {
            OpusAudioPlayer opusAudioPlayer = new OpusAudioPlayer(new File("test.opus"));
            opusAudioPlayer.play();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public OpusAudioPlayer(File audioFile) throws IOException {
        oggFile = new FileStream(new RandomAccessFile(audioFile, "r"));
        opusState = Opus.decoder_create(INPUT_BITRATE, 1);
    }

    private byte[] decode(byte[] packetData) {
        int frameSize = Opus.decoder_get_nb_samples(opusState, packetData, 0, packetData.length);
        int decodedSamples = Opus.decode(opusState, packetData, 0, packetData.length, decodeBuffer.array(), 0, frameSize, 0);
        if (decodedSamples < 0) {
            System.out.println("Decode error: " + decodedSamples);
            decodeBuffer.clear();
            return null;
        }
        decodeBuffer.position(decodedSamples * 2); // 2 bytes per sample
        decodeBuffer.flip();

        byte[] decodedData = new byte[decodeBuffer.remaining()];
        decodeBuffer.get(decodedData);
        decodeBuffer.flip();
        return decodedData;
    }

    public void play() {
        int totalDecodedBytes = 0;
        try {
            SourceDataLine speaker = AudioSystem.getSourceDataLine(audioFormat);
            speaker.open();
            speaker.start();
            for (LogicalOggStream stream : (Collection<LogicalOggStream>) oggFile.getLogicalStreams()) {
                byte[] nextPacket = stream.getNextOggPacket();
                while (nextPacket != null) {
                    byte[] decodedData = decode(nextPacket);
                    if(decodedData != null) {
                        // Write packet to SourceDataLine
                        speaker.write(decodedData, 0, decodedData.length);
                        totalDecodedBytes += decodedData.length;
                    }
                    nextPacket = stream.getNextOggPacket();
                }
            }
            speaker.drain();
            speaker.close();
            System.out.println(String.format("Decoded to %d bytes", totalDecodedBytes));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Answer 2

Looking to your code, I assume you missunderstood the meaning of "frame length". You are taking the number of bytes, but the frame length depends directly on how the file was encoded.

An audio file recorded at 48000 Hz has 48000 samples per second. This audio sample is usually a 16-bit integer (2 bytes), what means that you will have 48000 * 2 bytes per second in the non-encoded form (PCM-WAV).

An audio encoder like the opus will take multiple audio samples at once and encode them in a package. THIS is the frame. At 48 kHz these values could be for opus 120, 240, 480, 960, 1920, and 2880.