Colossyan's API offers a programmable video creation service. It allows you to generate videos at scale. To facilitate this, we provide a REST API using standard JSON as request and response payload format. This way you can integrate our service into any technology stack.

Features

• 💬 Over 600 voices

• 🌐 Over 70 languages

• 👄 Use any actor of your choice

• 👤 Create your own avatars through the API, and use them at your will

• ⚖️ Generate template videos at a massive scale, easily

• 📹 Everything that you can generate in the Colossyan Studio you can also generate via our API

  • 🖼️ Embedded images

  • 🎬 Embedded videos

  • 🎶 Embedded audios

  • 🔤 Embedded texts

• 📞 Get notified immediately via a callback mechanism

• 💅 Thanks to our web-based templating and payload exporting options it's incredibly easy to get up-n-running.

Jump right in

Constructing request bodies for complex video generation jobs can become very complicated very quickly. To get ahead of this we provide a way to use our web-based video-generation tool (Colossyan Studio) to export the payload for the video you see.

1. To do this, open any draft you'd like to generate via the API.

2. Click on the "Generate" button on the top right corner of your editor, than in the dialog appeared you should see a button called "Export to API", click that button.

   1. Note that you can only see this button if you have API access.

3. In the appearing modal, you should see the request body that you can send to generate the exact video that you can see in the editor.

4. Feel free to customise this payload.

Stable features

https://app.colossyan.com/api/v1

Experimental features

https://app.colossyan.com/api

Using the API, you can list the following assets:

• List avatars

• List voices

To create a new "instant" avatar please see the following page:

• Create avatar

There are multiple ways to generate a video.

• You can generate a video based on a template (that you create). In this case you only send us a reference to the template and a list of dynamic variables that you'd like us to replace in the template. See more here: Generating using a template

• You can generate a video, by sending a whole video-generation-job descriptor JSON to our API. See more here: Generating a video manually

Once a video generation has been triggered, you have multiple options to get notified once it's ready. See more here: Receiving a generated video

Throught the API it is possible to retrieve or delete a video that was already generated.

• Retrieve a video

• Delete a video

A video generation job holds data respective the video generation.

For example by retrieving the generation job you can tell which status the video generation is currenlty in. (e.g. generating, finished, failed)

• Retrieve video generation job

• Delete video generation job

In general, we always recommend to generate videos either by:

• Extracting the video-generation-job payload from our web-based UI editor. More details here: Extracting request bodies from the web-editor.

• Or by Generating using a template

If for some reason either of these options wouldn't be enough to achieve what you are after, in the following pages you can read about some advanced concepts, when working with our API.

Dynamic template variables are placeholders within your video script or scene text element that can be replaced with different values each time a video is generated.

They allow you to customize content dynamically without modifying the base template.

Learn more on the following pages on how to use them:

• Script template variables

• Text template variables

Follow the next steps to generate your first video using Colossyan's API.

1. Grab your token as described in the Authentication page.

2. Send your first request

const token = "<paste-your-token-here>";
const api = "https://app.colossyan.com/api/v1";

const job = {
  videoCreative: {
    settings: {
      name: "My first video",
      videoSize: {
        width: 1920,
        height: 1080
      }
    },
    scenes: [
      {
        tracks: [
          {
            type: "actor",
            position: { x: 420, y: 0 },
            size: { width: 1080, height: 1080 },
            actor: "karen",
            text: "This is my first video generated with Colossyan! Amazing!",
            speakerId: "aquXcfLbkxpW4BBI5qKm"
          }
        ]
      }
    ]
  }
};

const response = await fetch(`${api}/video-generation-jobs`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${token}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify(job),
});

const result = await response.json();
console.log(result); // { id, videoId }

1. If you successfully sent the request, you should have received two things in return.

   1. An id that you can use to delete or request status updates on the job

   2. A videoId that you'll be able to query the generated video with once it's ready.

2. Navigate to the workspace where you have created your API key. You should see the video generate. Once it's ready, it should look exactly like this

1. Congratulations! 🎉 You just generated your first video using our API!

Colossyan's API is using Bearer Authentication. This means, that every request has to be authenticated by sending the token in the Authorization header pre-pended by the text Bearer . See the example below:

const response = await fetch(`${api}/video-generation-jobs`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${token}`, // Authentication
    "Content-Type": "application/json",
  },
  body: JSON.stringify(job),
});

You can create or find your existing tokens at the bottom of the Workspace details tab in the Settings page.

The endpoint will return the available avatars for your workspace.

The types of avatars the endpoint will return:

• Studio: Avatars provided by Colossyan.

• Scenario: Also provided by Colossyan, but shown with a specific scenario background.

• Instant: Custom-made avatars. To learn how to create one using the API, check out the following page ⇒

• ssa_lite/ssa_studio: These are also custom-made avatars. To learn more, visit .

There are two different inference models supported by the API (returned by the supportedInferenceModel parameter):

• legacy

• neo

There are three different avatar variants supported by the API (returned by the supportedActorTrackVariant parameter):

• full_body

• bubble

• shoulder

Voices

To generate a video based on a template:

1. Head over to Colossyan and create your desired video draft using our web-based studio

2. Once you are done, click on the "Generate" button on the top right corner of your editor, than in the dialog appeared click "Export to API"

1. On the top right corner of the appearing dialog click "Save job as a template" and make note of the appearing ID. This is the ID of your template job, you can send this id with your request to start generate the video.

1. Now you have everything to send us a video generation request using the API endpoint below.

You can generate a video by sending us a video-generation-job descriptor JSON. This is essentially a recipe for a video. Constructing such a recipe manually for complex videos can become tedious very quickly. Due to this we strongly recommend to either:

• Extract the video-generation-job payload from our web-based UI editor. More details here: Extracting request bodies from the web-editor

• Or to generate based on a template. More details here: Generating using a template

Important concepts

To better understand the API, it's recommended to get an understanding of the following building blocks first:

• Scenes

  • A scene is a logical consolidation of related content.

  • A video can consist of multiple scenes. These scenes are concatenated together. Think of them as slides on a presentation. Each scene is meant to aggregate and encapsulate related content.

Here you can see the details of the endpoint below.

Deleting a video generation job will stop the video generation processing.

You can create an "instant" avatar by sending an image or video link to the API. The avatar will be generated based on the provided media.

The API will return the avatar's name, which can then be used for generating video with the avatar.

You can create dynamic template variables in the script on any scene, by simply typing a variable name between curly braces at any point of the script.

For example in the following script...

,,,

Dear {name}, I have seen that you haven't logged into your account for the past two months. Should we catch up quickly? Since then we released several cool stuff, such as {cool_stuff_1} and {cool_stuff_2}. Let me know if you have a free slot on your calendar next week!

,,,

...you have created three dynamic variables:

1. name

2. cool_stuff_1

3. cool_stuff_2

You can replace these template variables with their values, by sending their values in the request body when generating a video. The API will replace the placeholders in the script with the corresponding values you provide.

For example:

On how to generate a video you can find more information here:

You can use dynamic template variables in any text element, by simply typing a variable name between curly braces.

Take the following example:

,,,

Hey {name}, we’ve been improving!

Our newest feature, {feature_name}, is live!

,,,

In this text we created two template variables: name and feature_name .

You can replace these template variables with their values, by sending their values in the request body when generating a video. The API will replace the placeholders in the text with the corresponding values you provide.

For example:

On how to generate a video you can find more information here:

Upon posting a new video generation job, you have the opportunity to add a callback and some callbackPayload to the job. When the job is successful our service will issue a POST request to the url you provided in the callback field.

In the body of this post message, we add the following fields:

Timing

Scene duration

You can set the duration of each scene in three different ways. In either case no matter how long the duration of the tracks within the scene are, everything longer than the scene itself will be cut out.

Explicit scene duration

You can explicitly define the scene's duration in milliseconds by setting the duration property on it.

const job = {
  videoCreative: {
    settings: {
      name: "Simple video with a single scene",
      videoSize: { height: 1080, width: 1920 },
    },
    scenes: [
      {
        name: "Scene with explicit duration",
        duration: 2000, // ms
        tracks: [
          {
            type: "actor",
            position: { x: 200, y: 200 }, // From the top left in pixels
            size: { height: 680, width: 1520 }, // In pixels
            text: "This scene is exactly two seconds long. It does not matter how long any of the tracks are.",
            speakerId: "en-GB-RyanNeural",
            actor: "ryan",
          },
        ],
      },
    ],
  },
};

Referring to a track

If you would like your scene to be exactly as long as one of the tracks in it, you can do so by referring to the track on the scene. In this case Colossyan will try to figure out the length of the track in isolation first, and if it can do so, it'll assign the same duration to the scene itself.

Use the intrinsicDurationTrackReference property in the scene to refer to the referenceId of the track you would like the scene to have an equal duration with.

const job = {
  videoCreative: {
    settings: {
      name: "Simple video with a single scene",
      videoSize: { height: 1080, width: 1920 },
    },
    scenes: [
      {
        name: "This scene lasts until the actor stops speaking",
        intrinsicDurationTrackReference: "speech",
        tracks: [
          {
            type: "actor",
            referenceId: "speech",
            position: { x: 200, y: 200 }, // From the top left in pixels
            size: { height: 680, width: 1520 }, // In pixels
            text: "It does not matter how long I am talking. The scene will last as long as necessary.",
            speakerId: "en-GB-RyanNeural",
            actor: "ryan",
          },
        ],
      },
    ],
  },
};

Combining this technique with timing the tracks by referring to the scene's start and end time is a very powerful combination, that allows you to achieve intricate videos with the flexibility of handing varying track lengths.

If you do not explicitly set the duration, nor refer to a track within the scene, Colossyan will try to infer the best guess for the duration of the scene by looking at the tracks, and trying to find a single one with either an intrinsic or explicitly set duration.

This is only possible if there is only one track in the scene with a default or explicitly set duration. Otherwise Colossyan will throw a schema validation error, when queueing the job.

Track duration can be set in three different ways. No matter which way you chose, if the scene is shorter, the track will be cut to fit it.

If however the track you provided does not last long enough to reach the duration defined by you (for example a video ends earlier, or the actor finishes speaking), the track will disappear earlier than you defined.

Similarly to setting a scene's explicit duration, you can set the duration of a track by setting the duration property on it. The duration should be set in milliseconds.

You can set the track to start after a certain amount of time has passed since the beginning of the scene, and to last until a certain amount of time before the end of the scene. By setting both of these variables, you are essentially defining the duration of the track as well.

You can define these gaps from the start and end of the scene by setting the startTimeGap and the endTimeGap properties on the track.

If you do not define the duration of the track in any way, Colossyan will try to fall back to the intrinsic duration of the track if there is any.

Only video and actor tracks have intrinsic durations.

• In case of a video track the intrinsic duration equals the duration of the video itself.

• In case of an actor track the intrinsic duration equals the duration of the lipsynched video of the actor speaking.

To time when the track should start or end within the scene, you can use the startTimeGap or the endTimeGap properties on the track.

• Use startTimeGap and endTimeGap to make videos that work well for dynamic content.

• Refer to the track's duration in scenes to deal with dynamic track lengths.

• Do not over-constrain the timing of your scenes and tracks. This will yield in a schema validation error.