<img src="https://certify.alexametrics.com/atrk.gif?account=bIEZv1FYxz20cv" style="display:none" height="1" width="1" alt="">

Make a DJ Bot

 

Want to liven up your virtual Space with some MP3s played by a robot? Within 30 minutes, you can make that happen. The DJ Bot you'll have created by the end of this guide will have the human users of your application dancing all night long.

Project Complexity: ★★☆☆

In this guide, you will learn how to:

  1. Sign up to use the Spatial Audio API
  2. Create a High Fidelity Spatial Audio App and a Space to use with our DJ Bot
  3. Set up your development environment for writing the DJ Bot
  4. Write some DJ Bot code!
 

 

1. Sign Up to Use the Spatial Audio API

If you already have a High Fidelity Developer Account, you can jump straight to the next step.

Before your app can make use of the Spatial Audio API, you'll need a High Fidelity Developer Account. Sign up for an account or log in here.

 

2. Create an App and Space for the DJ Bot

If you already have a High Fidelity Developer Account, you can jump straight to the next step.

To authenticate our DJ Bot with your Space, you'll first need to create an App and a Space. To do that:

  1. Log in to your High Fidelity Developer Account.
  2. Under "New App Name", enter an app name.
    • Example: "My Test App"
  3. Click the "Create New App" button.
  4. On the page that appears, under "New Space Name", enter a space name.
    • Example: "My Test Space"
  5. Click "Create New Space".

Do not share the data you see on this page. It allows users to make authorized connections to your High Fidelity Spatial Audio Server.

 

 

3. Set Up Your Local Developer Environment

To complete this guide, you will need to install the following software on your computer:

Next, let's create a skeleton version of a NodeJS project we'll use for development:

Create a new folder on your local disk that you'll use for development, such as C:\code\hifi-dj-bot\. Then, install the JavaScript (NodeJS) version of the High Fidelity Spatial Audio Client Library by performing the following actions:

  1. Create a new folder on your local disk that you'll use for development, such as C:\code\hifi-dj-bot\.
  2. Using PowerShell or Terminal, cd into that development directory.
  3. Enter npm init to initialize a new NodeJS project, inputting data as prompted. The data you input isn't important for the purposes of this guide.
  4. Install the JavaScript (NodeJS) version of the High Fidelity Spatial Audio Client Library by typing npm install hifi-spatial-audio
      This will also automatically install the dependencies of the HiFi Client Library onto your system.

To make our lives easier later, we're now going to install all of the NodeJS dependencies used for this DJ Bot project. We'll make use of these dependencies in the next steps.

  1. Install audio-decode by typing npm i audio-decode and pressing enter.
    • When you install audio-decode, you may see some warnings or errors in your console window. These warnings and errors are benign, and you can safely continue.
    • This dependency lets the DJ Bot decode MP3 and WAV files.
  2. Install audio-format by typing npm i audio-format and pressing enter.
    • This dependency helps us create the necessary MediaStream object that we'll send up to the High Fidelity Spatial Audio Server.
  3. Install pcm-convert by typing npm i pcm-convert and pressing enter.
    • This dependency helps us create the necessary MediaStream object that we'll send up to the High Fidelity Spatial Audio Server.
  4. Install yargs by typing npm i yargs and pressing enter.
    • This dependency lets us more easily customize our DJ Bot on the command line when we run it.
  5. Install jose by typing npm i jose and pressing enter.
    • This dependency lets us securely generate a JWT used to connect to our Space.
  6. Install wrtc by typing npm i wrtc and pressing enter.
    • This dependency lets us create the necessary MediaStream object that we'll send up to the High Fidelity Spatial Audio Server.
    • If you see errors scroll by in your console after typing npm i wrtc and also see SKIPPING OPTIONAL DEPENDENCY, do not fret! The wrtc package has an optional dependency that may not build on your system. These errors are benign, and you may ignore them.

Inside your development directory, create a new file called index.js. This is where we'll write our code.

 

 

4. Write Some Code!

Now that you have your development environment set up, it's time to write some code!

Here's the entire contents of a nearly-completed index.js DJ Bot. Copy this code into your index.js file.



const yargs = require('yargs'); // Used to make it easier to parse command-line arguments to this script.
const crypto = require('crypto'); // Used to create a JWT associated with your Space.
const { SignJWT } = require('jose/dist/node/cjs/jwt/sign'); // Used to create a JWT associated with your Space.
const { MediaStream, nonstandard: { RTCAudioSource } } = require('wrtc'); // Used to create the `MediaStream` containing your DJ Bot's audio.
const fs = require('fs'); // Used to read the specified audio file from your local disk.
const path = require('path'); // Used to verify that the specified audio file is an MP3 or WAV file.
const decode = require('audio-decode'); // Used to decode the audio file present on your local disk.
const format = require('audio-format'); // Allows us to retrieve available format properties from an audio-like object, such as our `AudioBuffer`.
const convert = require('pcm-convert'); // Allows us to convert our `AudioBuffer` into the proper `int16` format.
import { Point3D, HiFiAudioAPIData, HiFiCommunicator, preciseInterval } from 'hifi-spatial-audio'; // Used to interface with the Spatial Audio API.

async function generateJWT() {
    // This is your "App ID" as obtained from the High Fidelity Audio API Developer Console.
    const APP_ID = "aaaaaaaa-1111-bbbb-2222-cccccccccccc";
    // This is your "Space ID" as obtained from the High Fidelity Audio API Developer Console.
    const SPACE_ID = "aaaaaaaa-1111-bbbb-2222-cccccccccccc";
    // This is the "App Secret" as obtained from the High Fidelity Audio API Developer Console.
    const APP_SECRET = "aaaaaaaa-1111-bbbb-2222-cccccccccccc";
    // Used to identify the DJ Bot client as the DJ Bot in your Space.
    const USER_ID = "DJBot";

    let secretKeyForSigning = crypto.createSecretKey(Buffer.from(APP_SECRET, "utf8"));
    let hiFiJWT;
    try {
        hiFiJWT = await new SignJWT({
            "user_id": USER_ID,
            "app_id": APP_ID,
            "space_id": SPACE_ID
        })
            .setProtectedHeader({ alg: 'HS256', typ: 'JWT' })
            .sign(secretKeyForSigning);
    } catch (error) {
        console.error(`Couldn't generate JWT! Error:\
${error}`);
        return;
    }

    return hiFiJWT;
}

/**
 * Play the audio from a file into a High Fidelity Space. The audio will loop indefinitely.
 *
 * @param {string} audioPath - Path to an `.mp3` or `.wav` audio file
 * @param {object} position - The {x, y, z} point at which to spatialize the audio.
 * @param {number} hiFiGain - Set above 1 to boost the volume of the bot, or set below 1 to attenuate the volume of the bot.
 */
async function startDJBot(audioPath, position, hiFiGain) {
    // Make sure we've been passed an `audioPath`...
    if (!audioPath) {
        console.error(`Audio file path not specified! Please specify an audio path with "--audio "`);
        return;
    }

    // Make sure the `audioPath` we've been passed is actually a file that exists on the filesystem...
    if (!fs.statSync(audioPath).isFile()) {
        console.error(`Specified path "${audioPath}" is not a file!`);
        return;
    }

    // Make sure that the file at `audioPath` is a `.mp3` or a `.wav` file.
    let audioFileExtension = path.extname(audioPath).toLowerCase();
    if (!(audioFileExtension === ".mp3" || audioFileExtension === ".wav")) {
        console.error(`Specified audio file must be a \`.mp3\` or a \`.wav\`!\
Instead, it's a \`${audioFileExtension}\``);
        return;
    }

    // Read the audio file from our local filesystem into a file buffer.
    const fileBuffer = fs.readFileSync(audioPath),
        // Decode the audio file buffer into an AudioBuffer object.
        audioBuffer = await decode(fileBuffer),
        // Obtain various necessary pieces of information about the audio file.
        { numberOfChannels, sampleRate, length, duration } = audioBuffer,
        // Get the correct format of the `audioBuffer`.
        parsed = format.detect(audioBuffer),
        // Convert the parsed `audioBuffer` into the proper format.
        convertedAudioBuffer = convert(audioBuffer, parsed, 'int16'),
        // Define the number of bits per sample encoded into the original audio file. `16` is a commonly-used number. The DJ Bot may malfunction
        // if the audio file specified is encoded using a different number of bits per sample.
        BITS_PER_SAMPLE = 16,
        // Define the interval at which we want to fill the sample data being streamed into the `MediaStream` sent up to the Server.
        // `wrtc` expects this to be 10ms.
        TICK_INTERVAL_MS = 10,
        // There are 1000 milliseconds per second :)
        MS_PER_SEC = 1000,
        // The number of times we fill up the audio buffer per second.
        TICKS_PER_SECOND = MS_PER_SEC / TICK_INTERVAL_MS,
        // The number of audio samples present in the `MediaStream` audio buffer per tick.
        SAMPLES_PER_TICK = sampleRate / TICKS_PER_SECOND,
        // Contains the audio sample data present in the `MediaStream` audio buffer sent to the Server.
        currentSamples = new Int16Array(numberOfChannels * SAMPLES_PER_TICK),
        // Contains all of the data necessary to pass to our `RTCAudioSource()`, which is sent to the Server.
        currentAudioData = { samples: currentSamples, sampleRate, bitsPerSample: BITS_PER_SAMPLE, channelCount: numberOfChannels, numberOfFrames: SAMPLES_PER_TICK },
        // The `MediaStream` sent to the server consists of an "Audio Source" and, within that Source, a single "Audio Track".
        source = new RTCAudioSource(),
        track = source.createTrack(),
        // This is the final `MediaStream` sent to the server. The data within that `MediaStream` will be updated on an interval.
        inputAudioMediaStream = new MediaStream([track]),
        // Define the initial HiFi Audio API Data used when connecting to the Spatial Audio API.
        initialHiFiAudioAPIData = new HiFiAudioAPIData({
            position: new Point3D(position),
            hiFiGain: hiFiGain
        }),
        // Set up the HiFiCommunicator used to communicate with the Spatial Audio API.
        hifiCommunicator = new HiFiCommunicator({ initialHiFiAudioAPIData });

    // Set the Input Audio Media Stream to the `MediaStream` we created above. We'll fill it up with data below.
    await hifiCommunicator.setInputAudioMediaStream(inputAudioMediaStream);

    // `sampleNumber` defines where we are in the decoded audio stream from above. `0` means "we're at the beginning of the audio file".
    let sampleNumber = 0;
    // Called once every `TICK_INTERVAL_MS` milliseconds.
    let tick = () => {
        // This `for()` loop fills up `currentSamples` with the right amount of raw audio data grabbed from the correct position
        // in the decoded audio file.
        for (let frameNumber = 0; frameNumber < SAMPLES_PER_TICK; frameNumber++, sampleNumber++) {
            for (let channelNumber = 0; channelNumber < numberOfChannels; channelNumber++) {
                currentSamples[frameNumber * numberOfChannels + channelNumber] = convertedAudioBuffer[sampleNumber * numberOfChannels + channelNumber] || 0;
            }
        }

        // This is the function that actually modifies the `MediaStream` we're sending to the Server.
        source.onData(currentAudioData);

        // Check if we're at the end of our audio file. If so, reset the `sampleNumber` so that we loop.
        if (sampleNumber > length) {
            sampleNumber = 0;
        }
    }

    // Generate the JWT used to connect to our High Fidelity Space.
    let hiFiJWT = await generateJWT();
    if (!hiFiJWT) {
        return;
    }

    // Connect to our High Fidelity Space.
    let connectResponse;
    try {
        connectResponse = await hifiCommunicator.connectToHiFiAudioAPIServer(hiFiJWT);
    } catch (e) {
        console.error(`Call to \`connectToHiFiAudioAPIServer()\` failed! Error:\
${JSON.stringify(e)}`);
        return;
    }

    // Set up the `preciseInterval` used to regularly update the `MediaStream` we're sending to the Server.
    preciseInterval(tick, TICK_INTERVAL_MS);

    console.log(`DJ Bot connected. Let's DANCE!`);
}

// Define all of the valid arguments that we can supply to this script on the command line.
const argv = yargs
    .option('audio', {
        describe: 'An audio file path',
        type: 'string',
    })
    .options('x', {
        describe: 'X Coordinate of the bot for spatialized audio',
        type: 'number',
        default: 1
    })
    .options('y', {
        describe: 'Y Coordinate of the bot for spatialized audio',
        type: 'number',
        default: 1
    })
    .options('z', {
        describe: 'Z Coordinate of the bot for spatialized audio',
        type: 'number',
        default: 1
    })
    .options('hiFiGain', {
        describe: 'HiFi Gain for the spatialized audio',
        type: 'number',
        default: 1
    })
    .help()
    .alias('help', 'h')
    .argv;

// Let's dance! 🎶
startDJBot(argv.audio, { x: argv.x, y: argv.y, z: argv.z }, argv.hiFiGain);

We say "nearly-completed" above because you have to fill in your APP_ID, SPACE_ID, APP_SECRET, and USER_ID in the code. You gained access to this information in the "Create an App and Space" step above. For reference, you can find these pieces of information in the High Fidelity Audio API Developer Console.

 

‘After filling in the above information, in the PowerShell or Terminal window you've been using, type node -r esm index.js --audio <path to local audio file> and press enter to execute the code. You can get a list of all command-line options by typing node index.js -h and pressing enter.

If the DJ Bot is running successfully, you won't hear anything, but you will see DJ Bot connected. Let's DANCE! in your PowerShell or Terminal window. If you specified the SPACE_ID that corresponds to the JWT you created for your Simple Web app, you will hear audio after you start your DJ Bot then open your Simple Web app and connect to that space.

Conclusion

In this guide, you learned how to create a groovy DJ Bot for use in your High Fidelity Spatial Audio Space. Your DJ Bot takes an MP3 file from your local disk and broadcasts the audio from that file into your Space! 🎶

You learned that you can use the "Simple Web App" guide to build a Web app from which you can listen to your DJ Bot. Click here to go to that guide.

For detailed technical information about the High Fidelity Spatial Audio Client API, visit the API documentation.

If you have any questions or comments, please reach out to support@highfidelity.com.