Sunday, August 12, 2018

Streaming from isochronous USB devices using WinUsb


I was struggling to figure out how to do isochronous USB transfers using WinUSB, The documentation is accurate but sparse, and if you’re not already familiar with how the whole process is supposed to work, it can be maddening to figure out what you’re doing wrong. In order to help other people from having to go through this pain, I wanted to present a working example, as well as a conceptual understanding that made it useful and finally click for me.

First of all, information about isochronous USB transfers. First, the Microsoft documentation on how to transfer USB data: https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/transfer-data-to-isochronous-endpoints Although it’s written from the perspective of assisting someone developing a USB device driver using the WDF framework, almost all the concepts apply directly to WinUSB, which seems to basically be a thin wrapper around the WDF APIs, shunting them down to usermode.

The next link from BeyondLogic is also pretty good, https://www.beyondlogic.org/usbnutshell/usb4.shtml#Isochronous and gives a good high level overview of why one would use isochronous transfers.

For this example, I’m effectively using the Cypress EX-USB2 development kit, and the firmware from this application note: http://www.cypress.com/documentation/application-notes/an4053-streaming-data-through-isochronous-or-bulk-endpoints-ez-usb

Unlike the application note, I didn’t want to have to use Cypress’s proprietary USB driver and sample application because I didn’t want another thing to distribute and also because it seems that to use it your application must be written in managed code, which I didn’t want to use.

With that out of the way, here’s the conceptual understanding that made sense to me. All the above is in the context of receiving isochronous data from a high-speed peripheral, although other types of transfers are similar. Originally, there was just USB full-speed, and it transferred data in units of frames. Each frame is 1 millisecond long, and the sending and receiving of frames is managed by the USB host controller in hardware. It’s real-time and happens every millisecond, without requiring OS assistance.

Every frame (millisecond), the USB host controller asks the peripheral for data. In response, the USB device can transfer up to 1024 bytes in return. If the device doesn’t have that much data to send, no worries, it can send less every frame.

However, the whole idea behind isochronous transfer is that the OS ensures that there is guaranteed bandwidth available such that so long as nothing anomalous happens, the peripheral will always be able to return the amount of data it needs every frame. In order to do this, the OS ensures that there aren’t more isochronous devices using the bus than bandwidth available.

If a device truly needs to send 1024 bytes of data every frame, it’s simple. That’s the maximum amount of data that can be transferred over the link, and there isn’t room for any other isochronous transferring devices - they’re out of luck.

In practice, many devices will use less than the maximum and could run concurrent with other devices, if the OS knew how much of those 1024 bytes the device was going to actually use. This is information that the max packet size field of the USB descriptor provides. A device could specify 512 bytes, for example, leaving half the bandwidth available for another device.

That’s USB 1 full-speed. Every millisecond, a frame occurs, and the USB host controller asks all isochronous peripherals for their data. They promise not to send more than they specified in their max packet size field, although they may send less. If they frequently send less, it’s wasteful in that it prevents other devices from using that bandwidth.

Then came USB Super Speed with USB 2.0, increasing the link bandwidth 40x (from 12 Mbps to 480 Mbps). For backward compatibility with existing devices, USB 2 keeps the original terminology the same as USB 1, with frames of data happening every millisecond. To accommodate the increased data rate, it subdivides the frames into 8 microframes, each lasting 125 microseconds. The amount of data per microframe also increased: now a peripheral can send 3 times 1024 bytes of data. As before, it can may also send less per microframe.

As with USB 1, the peripheral tells the OS the worst-case bandwidth it’s going to need, so that the OS and host controller can ensure they reserve enough time on the bus; however, the mechanism for doing so is a little more complicated due to backward compatibility.

There are now two mechanisms for the peripheral to tell the OS it needs less than the maximum possible bandwidth. The first is the interval field of the descriptor. Unlike USB 1 where every frame the controller asked the peripheral for data, USB 2 lets the peripheral tell the controller to skip microframes and only ask the peripheral for data every other microframe, every 4th microframe, or every 8th microframe (by setting the interval field to 1, 2, or 4).

The second way is the MaximumPacketSize field of the USB descriptor, and the max packet size is still 1024 bytes (the same as USB 1 full-speed). Each microframe is considered to be capable of transferring up to three transactions (of up to 1024 bytes). This information is encoded bit-wise into that field so that bits 0 through 10 are the PacketSize (same as USB 1), but bits 11-12 are now the number of additional packets.

Effectively, the spec authors used the additional bits beyond those needed for the packet size to encode another field indicating whether the peripheral is going to send 1 transaction per microframe ([12:11] = 2’b00) or 2 ([12:11] = 2'b01) or 3 ([12:11] = 2b’10). I found this really confusing at first so hopefully this explanation helps.

So, if your device needed to transfer the maximum possible data, it would set the interval to 1 (ask for data every microframe), the packet size to 1024, and the number of transactions to 2b’10 (3 packets a microframe). Alternately, if you had the full speed peripheral used an example earlier which only needed to send 512 bytes of data every frame and you converted it to a full speed device, you could set the interval to 4 (only ask for data every 8th microframe), the packet size to 512, and the number of packets to 2’b00 (only 1 packet per microframe).

When using less than the maximum, there are often several ways to split up the bandwidth required, depending on how much you skip microframes and how many packets per microframe you use. It’s not clear to me whether its better to send a little bit of data every microframe or skip a bunch of microframes and then send it all at once from a bus utilization perspective. If you know, leave a comment. In some cases, the choice is decided for you by the buffer sizes of your peripheral. If like me you’re using the EZ-USB2 device, it only has 4kB total buffer size, so above a certain data transfer rate you can’t skip too many microframes or the buffers will overflow. But assuming you have enough space, I’m not sure which approach is best.

In any case, that’s the summary of how USB 2 full-speed isochronous transfers work at the specification level. Every millisecond there’s a frame. This is divided into 8 microframes. Every 125 microseconds, the controller processes a microframe, and if it’s your peripherals turn (as specified by the number of intervals to skip), the controller asks your peripheral for data. Your peripheral can send 1,2 or 3 transactions, each containing up to 1024 bytes of data.

Now let’s get into the way that the Windows WinUsb framework handles isochronous data.
Use the existing examples to obtain a handle to your device (https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/using-winusb-api-to-communicate-with-a-usb-device), and select the alternate interface which streams isochronous data.  (You’ll always need to do this, because the USB specification states that the default interface cannot be isochronous. Don’t forget to do this, via https://docs.microsoft.com/en-us/windows/desktop/api/winusb/nf-winusb-winusb_setcurrentalternatesetting , or WinUsb won’t transfer data.)


Now you’re ready to set up your isochronous data transfer. The way this works is that in your application, you allocate some memory for a data buffer into which you want WinUsb to transfer data. Tell WinUsb about it with https://docs.microsoft.com/en-us/windows/desktop/api/winusb/nf-winusb-winusb_registerisochbuffer . Unlike reading from a bulk pipe, you use this same buffer for all your data transfers and you tell WinUsb about it before you start. This allows it to tell the memory manager that it’s going to need physical pages for your virtual buffer, and that they need to be accessible from kernel mode, and that it must keep them where they are and not page them out or move them around. By making these guarantees, it can tell the controller to DMA transfer data into those pages of memory without going through these potentially time consuming steps every transfer and potentially missing data from the device.

Next, WinUsb works on the concept of frame numbers. There’s a counter that continuously increments every time the hardware starts processing a frame, and isochronous transfers are always scheduled against one of those absolute frame numbers. In order to avoid dropping isochronous data, you need to ensure you schedule a transfer on every frame number required by your device (every single frame if the internal is 1, or every 8th frame if the interval is 4).

As an application writer, you *can* manage this yourself by querying the current frame number from the hardware and scheduling your transfer far enough in the future that you don’t miss the frame you want, but soon enough that you don’t lose data from existing data streaming in from your device. I imagine there are scenarios where that housekeeping is required, but for me I found it it easier to use https://docs.microsoft.com/en-us/windows/desktop/api/winusb/nf-winusb-winusb_readisochpipeasap

The way this works is that it schedules the transfer as soon as possible, on whatever frame number is next up in the hardware when you make the call. If you were to just call this routine, wait for the data to return, then call it again, it’s likely enough time has passed in the various software work required that you would have missed the next microframe of data from your device. To avoid this, use overlapped I/O and queue up several transfers in sequence.

When the first transfer finishes, WinUsb can be told to automatically start streaming data into the next queued transfer with no interruption or missing microframes. The way this works is by setting the ContinueTransfer flag to TRUE when queuing up transfers after the first one. This is in effect using multiple transfers to set up a circular buffer of data which the hardware continuously streams data into, and conceptually looks something like this something like this:

char buf[NUM_TRANSFERS * TRANSFER_SIZE];
bool firstIo = true;
// Queue up the transfers
for (auto i = 0; i < NUM_TRANSFERS; i++) {
    WinUsb_ReadIsochPipeAsap(buf + i*TRANSFER_SIZE, !firstio);
    firstIo = false;
}

while (true) {
    for (auto i = 0; i < NUM_TRANSFERS; i++) {
        // Wait for each transfer in sequence to finish.
        WinUsb_GetOverlappedResult(i);
        // Process the data for this transfer.
        // data = buf[i];
        // Add it back to the queue of transfers for hardware to stream into.
        WinUsb_ReadIsochPipeAsap(i, true /*ContinueStreaming*/);
    }
}

NUM_TRANSFERS will need to be at least 2 so that there’s always a transfer ready to accept data while you’re working on the other one, but can be larger to smooth out jitter in your application data processing at the cost of increased latency. As long as you process the data faster than it's arriving from hardware, this goes on indefinitely with no missing data.

In practice it’s a little more complicated, stemming from the observation at the start that devices can transfer up to their stated maximum, but also maybe less as they see fit. For example, suppose you have a device which lists the maximum possible transfer size of 3x1024 bytes each microframe. WinUsb works on USB frames, so each time you ask it to transfer data you’ll need to ensure you ask for at least 8x3x1024 = 24kB of data (because that’s how much your device said it might transfer each frame, since there’s 8 microframes per frame per the USB high speed specification). One possibility is it transfers this maximum amount of data, and you get it all back in sequential order in your provided buffer.

It might also transfer less. Potentially nothing for some transactions or microframes. It would be nice if WinUsb handled this for you, concatenating the bits of data one right after another in your buffer, then just telling you at the end how much data in total was actually transferred. It doesn’t. What actually happens (probably at the hardware layer? let me know), is that it divides your buffer up into smaller chunks of however many bytes your peripheral said it could need per microframe (3072 in our case).

In effect, you can think of your buffer being represented to WinUsb as not a

char buf[NUM_TRANSFERS * TRANSFER_SIZE];
but rather

char buf[NUM_TRANSFERS][MICROFRAMES_PER_TRANSFER][MAX_BYTES_PER_MICROFRAME];

As the application author, you get to choose NUM_TRANSFERS and MICROFRAMES_PER_TRANSFER (so long as it’s big enough to hold an entire frame of data or multiples thereof, so 8*n, n>=1 in our case but could be less if your device has a larger interval value), however MAX_BYTES_PER_MICROFRAME is told to us by the peripheral’s descriptor.
Each time the controller asks the peripheral for data, it advances one of the last array entries, then fills into it as much as the peripheral returned. If there’s extra space at the end of that array entry, that’s your problem as the application developer to deal with.

In order to make it possible for you to deal with it, WinUsb returns a list of values indicating how full each of those microframes actually were with data, so you know how much to process out of each entry before moving on to the next.
As a result, that data processing step in the above pseudo-code actually looks more like:

char buf[NUM_TRANSFERS][MICROFRAMES_PER_TRANSFER][MAX_BYTES_PER_MICROFRAME];
int howMuchData[NUM_TRANSFERS][MICROFRAMES_PER_TRANSFER];
bool firstIo = true;
// Queue up the transfers
for (auto i = 0; i < NUM_TRANSFERS; i++) {
    WinUsb_ReadIsochPipeAsap(buf[i], &howMuchData[i], !firstio /*ContStreaming*/);
    firstIo = false;
}

while (true) {
    for (auto i = 0; i < NUM_TRANSFERS; i++) {
        // Wait for each transfer in sequence to finish.
        WinUsb_GetOverlappedResult(i);
        // Process the data for this transfer.
        for (auto j = 0; j < MICROFRAMES_PER_TRANSFER; j++) {
            char* data = buf[i][j];
            int dataLen = howMuchData[i][j];
            // process data to data + dataLen amount of data from this microframe
        }
        // Add it back to the queue of transfers for hardware to stream into.
        WinUsb_ReadIsochPipeAsap(i, true /*ContStreaming*/);
    }
}

If the rest of your application expects to receive the data as one continuous stream in a nice contiguous buffer, you’ll have to come up with some scheme, such as memcpying the chunks into a separate contiguous buffer.

That’s basically it as far as how to use WinUsb for isochronous data transfer goes at a conceptual level.

Finally, here’s a concrete example:
#include <SDKDDKVer.h>
#include <stdlib.h>
#include <stdio.h>
#define WIN32_LEAN_AND_MEAN // CM_* APIs
#include <Windows.h>
#include <winusb.h>
#include <usb.h>
#include <initguid.h>
#include <cfgmgr32.h>
#include <strsafe.h>
DEFINE_GUID(GUID_DEVINTERFACE_USBPer, 0x1ab4ce39, 0x4959, 0x4c97, 0x84, 0x02, 0x85, 0xa2, 0xe5, 0xa2, 0x6b, 0x8f);
#define XFER_SZ 10
#define NUM_XFERS 8
int main()
{
    // See https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/using-winusb-api-to-communicate-with-a-usb-device for opening the device.

    // Get the device path string.
    PWCHAR DeviceInterfaceList = NULL;
    ULONG DeviceInterfaceListLength = 0;
    WCHAR DevicePath[MAX_PATH];
    CM_Get_Device_Interface_List_Size(&DeviceInterfaceListLength, (LPGUID)&GUID_DEVINTERFACE_USBPer, NULL, CM_GET_DEVICE_INTERFACE_LIST_PRESENT);
    DeviceInterfaceList = (PWCHAR)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, DeviceInterfaceListLength * sizeof(WCHAR));
    CM_Get_Device_Interface_List((LPGUID)&GUID_DEVINTERFACE_USBPer, NULL, DeviceInterfaceList, DeviceInterfaceListLength, CM_GET_DEVICE_INTERFACE_LIST_PRESENT);
    StringCbCopy(DevicePath, _countof(DevicePath), DeviceInterfaceList);
    HeapFree(GetProcessHeap(), 0, DeviceInterfaceList);

    // Open a handle to the device.
    HANDLE DeviceHandle = CreateFile(DevicePath, GENERIC_WRITE | GENERIC_READ, FILE_SHARE_WRITE | FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED, NULL);
 
    // Initialize WinUsb against the device handle.
    WINUSB_INTERFACE_HANDLE WinusbHandle;
    WinUsb_Initialize(DeviceHandle, &WinusbHandle);

    // ... end example code from link above.

    // Tell WinUsb to use your ISO alt descriptor (1 in our case).
    USB_INTERFACE_DESCRIPTOR UsbAltInterfaceDescriptor;
    WinUsb_QueryInterfaceSettings(WinusbHandle, 1, &UsbAltInterfaceDescriptor);
    WinUsb_SetCurrentAlternateSetting(WinusbHandle, UsbAltInterfaceDescriptor.bAlternateSetting);

    // Get the bandwidth requirements from the peripheral.
    WINUSB_PIPE_INFORMATION pi;
    WinUsb_QueryPipe(WinusbHandle, 1, 0, &pi);
    WINUSB_PIPE_INFORMATION_EX pix;
    WinUsb_QueryPipeEx(WinusbHandle, 1, 0, &pix);

    // In our discussion, MaximumBytesPerInterval is the max bytes per microframe.
    auto TransferSize = XFER_SZ * pix.MaximumBytesPerInterval * (8 / pix.Interval);

    WINUSB_ISOCH_BUFFER_HANDLE handle;
    PUCHAR data = (PUCHAR)malloc(NUM_XFERS * TransferSize);
    WinUsb_RegisterIsochBuffer(WinusbHandle, pi.PipeId, data, NUM_XFERS * TransferSize, &handle);

    // In order to later wait for I/O to finish, need to associate event with completion.
    OVERLAPPED completions[NUM_XFERS];
    for (auto i = 0; i < NUM_XFERS; i++) {
        ZeroMemory(&completions[i], sizeof(completions[i]));
        completions[i].hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
    }

    // In our discussion, # microframes per transfer.
    auto PacketCount = (NUM_XFERS * TransferSize) / pix.MaximumBytesPerInterval;
    PUSBD_ISO_PACKET_DESCRIPTOR descr = (PUSBD_ISO_PACKET_DESCRIPTOR)malloc(sizeof(USBD_ISO_PACKET_DESCRIPTOR) * PacketCount);
    for (auto i = 0; i < PacketCount; i++) {
        ZeroMemory(&descr[i], sizeof(USBD_ISO_PACKET_DESCRIPTOR));
    }

    // Buffer to hold contiguous data.
    auto bufContiguousBytes = NUM_XFERS * TransferSize;
    PUCHAR bufContigous = (PUCHAR)malloc(NUM_XFERS * TransferSize);

    auto firstio = true;

    // Prime all I/Os for reading.
    for (auto i = 0; i < NUM_XFERS; i++) {
        WinUsb_ReadIsochPipeAsap(handle, i * TransferSize, TransferSize, !firstio, PacketCount / NUM_XFERS, &descr[(PacketCount / NUM_XFERS)*i], &completions[i]);
        firstio = false;
    }

    // As they complete, extract the data and re-queue.
    while (true) {
        auto offsetInDstBuffer = 0;
        for (auto i = 0; i < NUM_XFERS; i++) {
            ULONG NumBytes;
            WinUsb_GetOverlappedResult(WinusbHandle, &completions[i], &NumBytes, TRUE);

            auto startDescriptor = (PacketCount / NUM_XFERS)*i;
            auto endDescriptor = startDescriptor + PacketCount / NUM_XFERS;

            for (auto j = startDescriptor; j < endDescriptor; j++) {
                auto offsetInSrcBuffer = (i * TransferSize) + descr[j].Offset;
                memcpy(bufContigous + offsetInDstBuffer, data + offsetInSrcBuffer, descr[j].Length);
                offsetInDstBuffer += descr[j].Length;
            }

            WinUsb_ReadIsochPipeAsap(handle, i * TransferSize, TransferSize, true, PacketCount / NUM_XFERS, &descr[(PacketCount / NUM_XFERS)*i], &completions[i]);
        }

        bufContiguousBytes = offsetInDstBuffer;
        // Process data in bufContigous.
    }
}

Sunday, February 25, 2018

DIY Slingbox, the Hard Way

For a long time now I've wanted the ability to record over-the-air TV and watch it later, either on my computer, TV, or phone from somewhere else.

Great, you say! Products abound to serve this specific need. Just go buy something for 10's to 100' of dollars and move on with your life.

However, for various reasons the things I tried all came up lacking. First of all, years ago I'd bought a few USB TV tuner sticks online for this purpose, all of which professed to come with various TV recording programs. There was one that worked well, but it only ran on a Mac. All the others had the unhelpful habit of crashing and failing to record a program, typically the one I really wanted to watch.

After those failed attempts, I then set up a PC running Windows Media Center, which could utilize those USB TV tuner sticks. It worked well enough, but had a few problems, not the least of which was that it's deprecated and no longer runs on Windows 10 without hacks. However even leaving that aside, it recorded into this proprietary format that was difficult to something you could watch on your phone on the go. Furthermore, it also had the habit of crashing from time to time and failing to record my shows.

At this point it was becoming clear that either the USB sticks or their drivers were just flakey, and I should have just bought a more professional piece of hardware. However, I'm stubborn, and besides, restarting the program always made it work again: it seems like these programs could automatically try that instead of silently failing.

All of this culminated with this year's Winter Olympics. I wanted to be able to watch sports I was interested in, later when I got home from work or on the bus home, and skip the commercials. Also, the Olympics were already underway and I needed something fast.

Hence, the hack written up in this blogpost for how to cobble together a makeshift setup using only the free software tools VLC and ffmpeg.

Acquiring the stream

Over the air TV broadcast is sent as a MPEG2 transport stream, which is a container format holding one or more video, audio and text data. This stream of bits is wrapped in various error correction and synchronization bits, then sent out over the air using one of a variety of transmission formats. The exact details aren't particularly important, because the TV tuner and its driver handles decoding those signals from the air and giving you back the original bits of the transport stream.

There are practically innumerable programs for receiving this data from the TV tuner USB sticks, but for this article I'll talk about using VLC on Windows. VLC media player is an open-source Swiss army knife of a video player which can also transcode video and stream it across a network or to a file. Of particular relevance to this article, it can also open TV tuners on Windows and decode + display the video and audio contained in the transport streams they present.

So that's step one. Install VLC. Unfortunately, there's a bug in VLC versions 2.0 and greater where it no longer opens TV tuners, at least for my USB TV tuner sticks. As a result, you'll need something earlier; I use version 1.9.

Launch VLC, then select Open Capture Device from the Media menu. Change the capture device to TV - digital, and choose the transmission format used in your country. Here in the USA, this is ATSC. Finally, pick your TV channel.

This is where things get a little interesting. This isn't as simple as typing channel "5" into the frequency box, for two reasons. The first is that when the US transitioned to digital TV, most stations changed the channel they broadcast on, because reasons. In order to avoid having to teach people new channel names for their favorite stations, along with the fact that these numbers are much less memorable than the original ones (channel 5 in my area became channel 48 -- doesn't quite roll off tongue), the new standard allows stations to advertise "virtual" channels which are shown to the user. When the user selects virtual channel 5, the TV actually tunes to channel 48. In order to find the actual channels, the first time you set up a TV in a new location, it performs a time-consuming scan of every single possible channel to see if there's anybody there. Hacky, but such is the story of backwards compatibility.

Normally, your recording software does exactly the same thing for you, but with this setup you'll have to do it manually. If you know the call letters of the station you want to watch, the easiest way is to search for call-letters-TV in Google and look at their Wikipedia page. For example, the aforementioned channel 5 in my area is KING. So, searching for KING-TV brings up their Wikipedia page. On the sidebar, the digital channel (the actual channel) will be listed.

If, on the other hand, you don't know your station's call letters, you've got a few choices.
One is to replace your city in this Wikipedia template and see if they have a list of stations compiled for your area. Alternately, go to TV Fool and put in your address, and it'll tell you the list of stations you can receive in your area, both the real channel and the virtual one.

With the real channel of your station known, there's one additional step. You still can't just type that channel in the box, because a "channel" is actually just another affordance for people to remember. Each channel actually maps to specific (in the USA) 6 Mhz wide swath of frequencies. Luckily, VLC knows this mapping, but you have to do a little work to get it. Click Advanced Options button and scroll all the way to the bottom, and put this channel in the box for Physical Channel.

That's it. Click Play and within a few seconds hopefully you're watching TV.

If nothing shows, up, there's a variety of things that could be going wrong. The first thing to try is to go to the Tools menu and choose Messages. Change the Verbosity to 2. Then, click the stop button, wait a few seconds, then click play again. Look through the log for any errors and see if any of them seem actionable. Admittedly, this is probably unlikely, but you never know. At this point, there's basically 3 things that can be going wrong:
  1. Your TV tuner doesn't have drivers installed.
  2. The TV signal is too weak to be received properly.
  3. VLC doesn't support your TV tuner.
To check for 1, open Device Manager by searching for it in the Start Menu and see if there are any unknown devices. If so, see if they go away when you unplug the tuner. If they do, you'll need the drivers. Google around for the model, or maybe if you're lucky it came with a CD.

That done, see if the problem is a weak signal. Try another strong channel, and fiddle with the antenna. If that doesn't work, and a nearby TV can receive the signal just fine while your stick can't, I don't know how to help you. :-/

Now that you've verified you can receive data from the tuner, it's time to figure out which sub-channel you're interested in. You've interacted with sub-channels when watching regular TV, but might not have known that they were called that. They're when you have channel 5.1 which is what you wanted to watch, and 5.2 and 5.3 which show old movies you're not interested in or infomercials.

Most probably, VLC will have chosen the first sub-channel when it started playing from the tuner. You can change channels from the Playback menu under the Programs submenu.

Unfortunately, in the MPEG standard, these sub-channels, like everything else, just another user-friendly mapping to an underlying technical value, and VLC deals only with the technical value. In this case, these are specified in a set of tables collectively called the program-specific information (also discussed by the standards committee). There are too many details not relevant to this discussion to go into deeply, but at a high level, a Virtual Channel Table associates a sub-channel (called the minor channel) with another arbitrary number, known as the program number. This number, in turn, is used to identify the video, audio, and closed caption (CC) streams of data associated with that sub-channel through yet more tables of indirection.

We need to determine the program number associated with our desired sub-channel (probably channel .1). Luckily this is easy: it's just what VLC lists as the number when you change programs via the aforementioned menu. Change to the program you wanted to watch (the lowest number probably) and remember it.

With that information, we now have what we need to have VLC record a single channel to disk for later watching. To do so, we'll use the lightly documented various magic boxes. In particular, data comes into VLC from the TV tuner. From there, we'll use the duplicate module. Not to duplicate anything, but because as part of duplicating it contains functionality to select only a single program from the transport stream. Then, we'll use the file output method of the dst option of the std output module to save that data to a file.As part of saving, VLC needs to know how to wrap up the various video, audio and CC data we've selected into a single file. The default option is to have VLC wrap it up in another transport stream, just like it came to us off the air, and that's fine for our purposes so we'll set the mux option of the std output module to ts.

Put together, this looks like:

:sout=#duplicate{dst=std{access=file,dst=c:\\tv\\tv.ts,mux=ts},select="program=3"}

Finally, we need to tell VLC to use the TV tuner, and what channel to tune it to, which we can do like so:

atsc://frequency=0000 :dvb-physical-channel=48

Try running this command on your channel from earlier for a few minutes before exiting VLC, and you should see several megabytes of data in the file specified:

vlc.exe atsc://frequency=0000 :dvb-physical-channel=48 :sout=#duplicate{dst=std{access=file,dst=d:\\tv\\tv.ts,mux=ts},select="program=3"}

Transcoding

If the only goal is to be able to watch a show later on the same computer from which you recorded it, you're done. Well, not completely done in that it doesn't automatically start or stop recording, but VLC will happily play back what it just saved. However, one of my main goals of this hack was to be able to watch what was recorded later on the go, in particular on my iPhone. For that to work, the information we just saved needs to be transcoded, for two reasons.

First, it's way to big. ATSC allows up to TV stations to transmit data at around 20 Mbps, faster than most phones will stream while on a cellular network or someone's slow WiFi. We need to make it smaller. Second, and more troubling is that at least in countries which broadcast with the ATSC standard, the video and audio are encoded in a format that most mobile phones don't support. That's because what's broadcast is MPEG2 video and Dolby AC3 audio, whereas most phones can only decode MPEG4 H264 video and AAC audio. Furthermore, many channels broadcast interlaced video (if you've see 1080i as the resolution of a channel on your TV, the I stands for interlaced), and phones can typically only play progressive video. Consequently, the original video also needs to be deinterlaced, at which point it'll need to be re-compressed anyway, so we might as well transcode it into a newer smaller format.

To do all of this, we'll use another open-source tool called ffmpeg. If VLC is swiss army knife of playing back and saving video, ffmpeg is the equivalent knife for transcoding video. It can read almost any video or audio format and the same for writing to them. (Actually, both VLC and ffmpeg share a common library, libavcodec, from whence they obtain most of their functionality.) The upshot of this flexibility is some complexity, combined with at times missing documentation. To be fair to the ffmpeg team, this has improved greatly in recent years.

Format Choice

The main goal of this project was to make it such that I and my friends could watch the Olympics from our phones or laptops at work on the commute there and back. As it happens, I have an iPhone, and most of my friends and family only have Mac laptops. As it goes with user-friendly design, I wanted to minimize the friction for them to use whatever I came up with, which to me meant absolutely no installing any apps, either on the phone or the laptop. Also, no requirements to download an entire video file before watching. I wanted it be as simple as streaming a video on YouTube, which also meant that the file size needed to be small enough that it would reliably stream on a mobile network and on mediocre internet connections. In particular, the requirement to be able to stream to an iPhone limits the choice of video and audio compression formats to effectively H264 and AAC audio. Theoretically since last year one can also use H265 video compression, but I didn't explore this.

FFmpeg has an informative page on the various options for H264 compression, including a list of which profiles and levels are supported by which mobile devices. These days if you're targeting iPhones, there's no need to use anything but the highest level, as phones which only support lower levels are all no longer supported by Apple. Also important is the option to use the +faststart option, which stores metadata about the file at the beginning so that playback can begin as soon as it starts downloading. Another important point to note is that iPhone doesn't support interlaced playback, so the raw data from over the air will need to be deinterlaced through some mechanism before the phone will play it back. Finally, I was targeting watching the result on a regular iPhone which has effectively 720p resolution. Since the signal comes in at 1080i , we could resize it to match the iPhone's screen before transcoding it to make a smaller file.

In addition to ensuring the file is as compact as possible and matched to playback on the iPhone, there was also the desire that it could be encoded fast enough that it was done by the time the next TV program came on. For simplicity I decided to just let the machine stream the NBC channel 24 hours a day so that I didn't have to manage figuring out exactly when what was worth watching was on, so this was needed to ensure there wasn't an endless stream of files backing up never finishing encoding. However, there was also the desire to be able to watch whatever was happening relatively close to live, which is easier the faster the recording encodes.

For managing that, there's the preset option which trades off speed for file size, for a given quality. I found that the default medium worked well enough for my relatively older AMD desktop machine, but on the lower powered Atom PC I wanted to use for encoding, I needed to change it to veryfast to keep up.

Finally, I found the default quality factor was a little too conservative, and I could reduce the quality to 26 without noticeable degradation watching on my phone.

Audio is relatively simpler (or at least, I didn't investigate it as much). There's also a page on this but I mostly stuck with the default options and left it be.

Putting all this together, the FFMPEG command line to accomplish all this comes out to be:
ffmpeg -i recording.ts -vcodec libx264 -preset veryfast -profile:v high -level 4.2 -s hd720 -acodec aac -ac 2 -ar 48000 -ab 196k -deinterlace -movflags +faststart encoded.mp4

The general ffmpeg documentation is pretty good at explaining the various options, but a little overwhelming, so breaking down each section:

-i tells ffmpeg the input file, and comes before any of the conversion options.

-vcodec libx264 -preset veryfast -profile:v high -level 4.2
comes from the H264 encoder page following the instructions for quality, phone profile support and encoding speed.
-s hd720
tells ffmpeg to resize the video to regular HD (720p) size
-acodec aac -ac 2 -ar 48000 -ab 196k
also comes from the AAC page. I change the sample rate to 48000 since that matches the incoming stream, and the bitrate to 196k as I found I could hear compression artifacts using 128k.
-deinterlace
tells ffmpeg to use its default deinterlacer to convert the incoming video to progressive video before resizing, compressing, etc, and is needed because the video comes in as interlaced.

As an aside, if you leave out the deinterlace option, ffmpeg will still run and produce a video. It's just that in scenes with motion, you'll end up with interlacing artifacts in the output file. That's because without being told otherwise, ffmpeg will just smush the two fields of the interlaced video one after another into a single frame of progressive video. This ends up working fine when presented with a still image, but in a scene with lots of motion, the two fields will contain very different pictures and the smushed frame will be full of scissor lines.

Finally, -movflags +faststart
This adds to the flags ffmpeg uses for post-processing and tells it to move the metadata about the file to the beginning so the phone can immediately start decoding the video as it downloads. This is required because normally this information is written to the end, after all the file is encoded.

Watching Remotely

So now we have the ability to record some show from over the air and convert it to something that the phone will play. Now what's needed is making it available remotely for watching on the go. Since we're using a Windows machine in this guide, the easiest way to do this is to use the built-in Internet Information Server to host a web site on this computer. There are various guides, but I couldn't find anything that listed all the steps required in a single step. Consequently, take a look at a previous post I wrote up on how to do this.

Putting It All Together

With all the basic pieces in place, the final step is to automate it so that you're not sitting there manually clicking record, running ffmpeg, and copying the files to the web server directory.

I set this up using two PCs. One, an old laptop, sits near the window with an unobstructed view of the TV tower. There, I used a small Python script to start a new recording every half an hour. This PC also hosts the web server which serves out the encoded files. I configured Windows to share the volume which stores these two folders over the network.

On a second PC, I used another Python script to watch for new files and launch ffmpeg to encode them.

On the recording PC, the script looks like this:

On the encoding PC, the script looks like this:

Then, on a second PC (my main desktop).

Running the actual encoding is done with a batch file, which just contains the live above directing ffmpeg to transcode the file.

Running a web server and sharing files from your home PC using Internet Information Server

If you're using a Windows computer, you already have everything you need to host a web site, and share files from your computer to anyone on the Internet, without installing anything. I couldn't find a simple guide for setting this up, hence this post.

Step 1: Enable the Internet Information Server feature.

The simplest way is to search for "Turn Windows features on or off", then click the Internet Information Services checkbox and click Ok.

Step 2: Create a directory you want to share

For this example, we'll use a folder called shared_files in your Documents folder.

Step 3: Add the directory to your website

Upon enabling the IIS feature, Windows configures a single Default Site which serves files and subdirectories out of c:\inetpub\wwwroot. However, you can add Virtual Directories which are served out of other locations on your computer, which is how we'll make the folder you created available online.

Start by launching the Internet Information Services (IIS) Manager:

Then expand the tree on the left-hand side until you get to Default Web Site, then click Add Virtual Directory:
Type in what you want it to appear as on your website. Here, we've chosen files, so you can get to it at http://yourcomputer/files. Then, click the button next to Physical Path and choose the directory you want to share and click OK:
Next, click Connect As, then Specific user. Type in your username and password. This is needed because IIS does not run as your user on the machine, but instead as a separate sandboxed account for security purposes which doesn't have permission to access files you created. So, you'll have to give it that ability by telling it what user and password it should use for this folder.
Finally, IIS does not show the files in a directory by default, also for security reasons. (In other words, you need to know exactly the HTML file you want to browse.) To change this, open the tree on the left so it shows the new Virtual Directory you created, and click on it. Then double click on the Directory Browsing icon in the main view, and on the right, click Enable:

Done

Finally, open up a web browser and navigate to your computer's IP address and directory you added, and you should see the list of files:

MIME Types

Finally, as a coda, you may find that IIS will give you an error when you click on certain files. If so, that's probably because you need to add that file type's extension to the list of MIME types IIS knows about. In the same place you clicked on Directory Browsing, double-click on MIME Types. Then, google for stackoverflow, the file extension you want to share, and the keywords mime type. You'll probably find the correct mime type, which you can add using that dialog.