Skip to content

Streaming (Audio and Video)

Jump to bottom Edit New page
Anthony Diamond edited this page May 1, 2023 · 11 revisions

Streaming Video

NOTE: This covers video streaming using a dedicated RTMP server, as well as an alternative for hosting one.

Video can be streamed and received using (Mostly open source) tools like 'ffmpeg', and 'nginx'. Compiled binaries can be found on the websites linked.

Several streaming solutions are available aside from 'ffmpeg', but they usually aren't oriented around zero-latency streaming. For example, OBS (Open Broadcast Software) is a great solution for normal video streams, but it tends to be problematic for this kind of setup.

'nginx' has specific versions with RTMP support. RTMP support will be needed to follow this recommended solution. For Windows machines, you can find compiled binaries here. For more information, please view the "Notable references" section.

An excellent guide for setting 'nginx' up as an RTMP server can be found on the OBS forums. There aren't necessarily special settings needed for 'nginx' after following that guide. You may want to specify a "chunk size", or "buffer length", but doing this should be optional.

Initial testing of 'nginx' for these purposes was done using the following settings:

chunk_size 1024; buflen 16ms;

Using those settings should not be required, and could be problematic. It's usually best to stick with the default settings.

Once you have an RTMP server running (Either on a dedicated machine, or on one of the computers participating), you'll need to stream to the server. Options are available with 'ffmpeg' to stream directly to individuals, but this segment does not cover this. In most situations where you have more than one person viewing the game, direct streaming isn't as viable.

Using 'ffmpeg', you can stream to the RTMP server using whichever commands you wish. The exact method of streaming depends on your situation. This segment covers 'DirectShow' streams/devices (And by extension, Dxtory) on Windows. This segment also only covers streaming with X264. The official builds of 'ffmpeg' should already support this. If you for some reason need an external distribution of it, you can find one here. Other encoders have yet to be documented. Using X264, we are able to encode and decode video from the RTMP stream.

Other streaming solutions, such as RTP and RTSP may also be used, but are not documented here. The official 'ffmpeg' streaming guide may help with streaming over RTP or RTSP. Point-to-point streaming may be ideal with two players. Point-to-point streaming solutions do not require third party tools. Although, you will still need 'ffplay' (Or similar).

Documented Situations:

  • On Windows using Dxtory (Proprietary software required):
    • Dxtory provides DirectShow "devices", which are usable from 'ffmpeg'. These are currently called "Dxtory Video X", where X is a number between 1 and 4. Example: "Dxtory Video 1"
  • On Windows using the generic screen recorder (Free for Windows users):
    • Generic screen recording can be done using the "screen-capture-recorder" device/stream. When using this stream, the "crop filter" may be useful. The filters are very useful if you're willing to learn how to use them. Crop filter example: "-vf crop=WIDTH:HEIGHT" (Where 'WIDTH' and 'HEIGHT' are user-defined)

If you are unsure about available DirectShow devices, you may use this command to enumerate them:

ffmpeg -list_devices true -f dshow -i dummy

DirectShow device names are case-sensitive, please keep this in mind. Non-DirectShow oriented examples have not been provided. Please consult the official documentation for further information.

Once you know what device you're going to use, you can either input the information yourself, or you can work off of this template:

ffmpeg -f dshow -i video="DSHOW_STREAM_HERE" -r FRAMERATE -s VIDEO_WIDTHxVIDEO_HEIGHT -threads THREAD_COUNT_HERE -vcodec libx264 -tune zerolatency -preset X264_PRESET_HERE -pix_fmt yuv444p -f flv rtmp://RTMP_SERVER_ADDRESS_HERE:RTMP_PORT_HERE/APPLICATION_NAME_HERE/STREAM_NAME_OF_CHOICE

How to fill out the template (For an example, view below):

  • DSHOW_STREAM_HERE:
    • This is the DirectShow device/stream as explained above. For more information, click here.
  • FRAMERATE:
    • This represents the framerate of the encoded video. Some players may have trouble playing abnormal framerates. It's best to just keep this at 30, or 60 if your internet connection can handle it.
  • VIDEO_WIDTH & VIDEO_HEIGHT:
    • This will be the resolution of the encoded video. Please supply the resolution using the correct syntax. Example: "640x480".
    • For the area of the DirectDraw stream/device used, please view the "crop filter" documentation. Using this is ideal when using the "screen-recorder" device/stream.
  • THREAD_COUNT_HERE:
    • The number of threads available to 'ffmpeg' and/or X264. For more information, please view the official documentation.
    • If unsure, either remove the "-threads" section, or set this to 0.
  • X264_PRESET_HERE:
    • This is the X264 preset; if you've used X264 before, then you know what this is.
    • If not, this page should help you pick a preset.
    • The exact preset you should use depends on the situation, and the processors available.
    • Initial testing was done using "faster" and higher; "veryfast", "superfast", and "ultrafast".
    • The "fast" and "faster" presets, paired with high-to-medium CRF values has yielded the lowest latency streams with further testing.
  • RTMP_SERVER_ADDRESS_HERE:
    • This should be the (IP) address of the server / RTMP host machine. This is not necessarily the same address as the IOSync host.
  • RTMP_PORT_HERE:
    • The standard RTMP port is 1935. That being said, you may use other ports besides this one.
    • Specifying a port (Separated by the ':' character) may not be necessary, if the RTMP server is being hosted using port 1935.
    • When hosting an RTMP server, you may currently use the same port-number as IOSync. This is because IOSync is currently UDP based, and RTMP is generally a TCP-based protocol.
    • This port must be forwarded/open for the RTMP-server's network (TCP). Opening the same port for IOSync may not count if you did not open it for TCP.
  • APPLICATION_NAME_HERE:
    • The name of the "application" on the 'nginx' server; this basically acts as a parent directory for streams.
  • STREAM_NAME_OF_CHOICE:
    • Unless your RTMP server has specific settings against this, this is user-defined.
    • Obviously, the only stipulation for using this file/folder/stream name is to use the same address for clients.
    • Once again, this depends on the server's settings. But, by default, you should be okay doing this with 'nginx'.
  • Other:
    • The portion specifying "-tune zerolatency" is important; as the argument-name implies, it reduces latency significantly.
    • The "animation" tuning may add latency; testing inconclusive.
    • The portion specifying the "pix_fmt" argument is there for the sake of maximum compatibility.

For extra information regarding X264 presets and configuration, please visit this page.

For the lazy, here's a basic template (Modify as you see fit; thread count may need to be changed):

ffmpeg -f dshow -i video="Dxtory Video 1" -threads 4 -vcodec libx264 -tune zerolatency -preset fast -crf 30 -pix_fmt yuv444p -an -f flv rtmp://127.0.0.1/live/test

The RTMP stream address may need to be changed. Also note this example's usage of CRF. For details, please visit the 'ffmpeg' H.264 encoding guide. Also keep the 'dshow' device in mind; you may need to use "screen-capture-recorder". Device/stream input depends on the platform; this example was written for Windows.

Once you've filled out the above template (Or wrote it yourself), use that command from the command-line in the "binaries" folder ("bin") of your distribution of 'ffmpeg'. Doing this should result in the stream beginning, and being broadcast to anyone who connects to the RTMP server.

Now that we've set up our stream, and an RTMP server, players need to actually watch it. Watching the stream can be done using a number of media players, for example, Media Player Classic and VLC have RTMP support. Using these media players may be beneficial, however, their options for reading RTMP streams aren't covered here.

For the sake of example, we'll be keeping things simple by using 'ffplay'. 'ffplay' is a tool (Commonly distributed with the 'ffmpeg' project), which allows you to playback audio and/or video. We'll be using it to read the RTMP stream.

Following the documentation for the previous template, this should be straight forward:

ffplay rtmp://RTMP_SERVER_ADDRESS_HERE:RTMP_PORT_HERE/APPLICATION_NAME_HERE/STREAM_NAME_OF_CHOICE -fflags nobuffer

The "-fflags nobuffer" segment is of interest, and it's one of the reasons we recommend 'ffplay'. The "nobuffer" flag keeps the player from buffering extra information before displaying what it already has downloaded. This allows us to stream with less of a delay to the player. Options for this may be available for other third-party media players. For more information on 'ffplay', click here.

As far as video is concerned, everything should be in working order.

For further documentation on 'ffmpeg', please read their official documentation.

Notable references:

Alternatives:

#Streaming Audio:# Audio streaming may be done separately, or through 'ffmpeg'. For the sake of example, I will be showing you how to stream audio with your video. Please view the "Streaming Video" section beforehand.

Following the example in the video section, we can add to the command-line arguments for the 'ffmpeg' binary specifically. First off, we'll want to specify an audio device/stream to 'dshow'. To do this, add:

:audio="AUDIO_STREAM_HERE"

to the end of the video-string (Next to the right quote). The device/stream you use as your 'AUDIO_STREAM_HERE' argument can be found by enumerating the DirectShow devices/streams available. As posted above, this can be done by using this command from the command-line:

ffmpeg -list_devices true -f dshow -i dummy

That will output a specific section for audio devices. If your intent is to output the system's default sound stream, you can specify "virtual-audio-capturer". This "capturer" is not always ideal, however. Third party solutions are available for virtual audio devices on Windows. Such tools will allow you to sandbox specific applications' audio playback. If you're using the system that's running the game/application you're streaming, this can be a necessity. For example, audio would be problematic using the "virtual capturer" if the host machine is being used to voice chat with the players who are listening.

Once you've picked your audio device, you need to ensure it can output the format you require. In the event you receive an error regarding the audio format, you may need to specify an audio codec. In this case, we'll be using 'libmp3lame'. For details on MP3 encoding, click here. To use this encoder, simply add the following after you insert the audio-stream segment:

-acodec libmp3lame -ar 44100

The "ar" argument specifies the sample-rate of the audio. That should be sufficient to stream audio. For more information on using 'libmp3lame', click here. In the event audio streaming causes video delay, external solutions may be necessary.

Notable pages:

Add a custom sidebar
Clone this wiki locally