uTasker USB audio capabilities

[Caleb]

Hi Mark, all,
   I'm starting a new project that needs to have multi-channel, full-duplex USB audio.  This will collect many microphone channels, as well as have simultaneous loudspeaker channels too.

Needs for the immediate project:
 16kHz sample rate
 9 channels of capture (USB IN, SAI RX via i.MXRT SAI1, with 2 rx data pins on 8 slots/pin TDM bus)
 2 channels of playback (USB OUT, SAI TX via SAI1 tx pin, TDM bus)

I believe this requirement can fit into a standard USB Audio Class 1.0 with no problem.  However, UAC 1.0 was crazy, and specifically seems to disallow high speed.  Not sure why that is...

Anyway, I'd like to use uTasker as a platform for USB audio going forward, with potentially *many* more channels in both directions, say up to at least 32in and 8 out or so, which will require USB Audio Class 2.0 to run on a high speed USB bus. 

So, my questions are:
* What's the state of the USB Audio class drivers currently (I will be building and testing momentarily to try things out, but I figured I'd get this question off into the ether)
* Will uTasker work well for the current project (UAC1.0 minimum project, multiple SAI channels)
* Will uTasker work well for the future projects (high speed USB).

Thanks,
  -Caleb

[Caleb]

I did just try compiling and ... it doesn't compile on i.MXRT1060 when I enable either USE_USB_AUDIO or USE_HS_USB_AUDIO.

Several errors, but as a generality:
```../usb_Audio.h:60:50: warning: "SAMPLING_FREQUENCY" is not defined, evaluates to 0 [-Wundef]
../usb_Audio.h: In function 'buffer_wrap':
../usb_Audio.h:60:50: warning: "SAMPLING_FREQUENCY" is not defined, evaluates to 0 [-Wundef]
   60 |     #define _PLL_COMPENSATION  (PIT_FREERUN_FREQ(SAMPLING_FREQUENCY)/200)// 0.5% adjustment
      |                                                  ^~~~~~~~~~~~~~~~~~
../../../Hardware/iMX/iMX.h:12338:46: note: in definition of macro 'PIT_FREERUN_FREQ'
12338 | #define PIT_FREERUN_FREQ(Hz)    ((PIT_CLOCK/(Hz)) - 1)
      |                                              ^~
../usb_Audio.h:61:9: note: in expansion of macro '_PLL_COMPENSATION'
   61 |     #if _PLL_COMPENSATION == 0
      |         ^~~~~~~~~~~~~~~~~
../../../Hardware/iMX/iMX.h:12338:45: error: division by zero in #if
12338 | #define PIT_FREERUN_FREQ(Hz)    ((PIT_CLOCK/(Hz)) - 1)
      |                                             ^
../usb_Audio.h:60:33: note: in expansion of macro 'PIT_FREERUN_FREQ'
   60 |     #define _PLL_COMPENSATION  (PIT_FREERUN_FREQ(SAMPLING_FREQUENCY)/200)// 0.5% adjustment
      |                                 ^~~~~~~~~~~~~~~~
../usb_Audio.h:61:9: note: in expansion of macro '_PLL_COMPENSATION'
   61 |     #if _PLL_COMPENSATION == 0
```

among other errors.  Is this expected to work?  Trying to compile for MIMXRT1060 EVK.

Thanks,
 -Caleb

[mark]

Hi Caleb

The USB audio document is at https://www.utasker.com/docs/uTasker/uTaskerUSB_Audio.pdf

Audio has been used in MIC -> speaker configurations and dual-microphone (without speaker) configurations - connected to ADC/DAC or codec (SAI like). It has been used on FS and HS controllers on the Kinetis parts (HS USB is compatible in the i.MX Rt parts).

It hasn't been used as a part of a composite device and I also have a problem when trying to build with USB-CDC and Audio enabled - it is necessary to disable CDC (then if finds SAMPLING_FREQUECY in usb_audio_descriptors.h).

As with any USB class the first thing that needs to be done for the required configuration is to set up the buffer descriptors - I have attached one for the 2 microphone case, whereby yours will be a lot more complicated (although you may have this already).

The main operation is to source and sink the isochronous buffers (as illustrated in the audio document) - when multiple such source and sinks are in operation these have to be done in parallel but each is effectively the same basic operation.

As noted above there are references for ADC/DAC and also SAI (on Kinetis) but each design will be different so the idea needs to be adapted to the actual interface(s) in question to implement the actual HW requirements.

Note that audio uses different types of synchronisation and the references use a PLL in SW approach and not an interrupt feedback channel. The best synchronisation technique also needs to be defined for the task in hand.

The HS USB controller in the i.MX RT 106x supports 8 endpoints (one for control and so 7 for the class interfaces) so the first thing to check is that this is adequate for the configuration, whereby IN and OUT can be shared on a single physical endpoint. Isochronous mode has been worked with at maximum speed (on HS controller) so the fundamentals are in place.

The references show general operation and so will not be directly suitable for your complex case with multiple interfaces so there would be work involved to:
- configure the descriptors and then configure the interfaces (endpoints) to match
- configure and attach the input/outputs accordingly (that is feeding output isochronous buffers attached to the endpoints and consuming received isochronous data) - typically this interrupt driver or, better, DMA driven and synchronised to the USB audio rate
- implement any synchronisation techniques required for the system or individual channels
- add any particular controls (like enabling/disabling/muting) for each of the channels

The USB drivers can handle the fundamental operations and there is a class reference that should essentially enable basic operation to be checked (but not necessarily hooked into a source/sink device of choice) so it will take extra work to configure and subsequently implement the full details of your quite complicated requirement.

Regards

Mark

[Caleb]

Ah, so sounds like it's not too straightforward.  I have run the sai demo from NXP, and I can actually get enough channels running, and they support UAC 2.0.  I would prefer to be a utasker project, but I also need to get it done quickly, so really need to start with a working example.

Thanks,
  -Caleb