Trying to get uTasker to work with my Teensy 3.6 board

[creitzel]

Mark,

First let me give the good news.  I just tried to do a build with FS USB MSD and SDCard support, using the changes you made to the code, and it works.  I can see my SD Card, and I can even stream MP3's off of it, to both my PC and to my car stereo.  :-)

The K66 has a new PLL dedicated to generating a 480MHz USB PHY clock, which however needs the VBUS to be connected to work.

Since I haven't used it in a real product yet I have presently accepted that the PLL will not work unless the HS USB is connected to a host. In fact, the 'final' and 'complete' solution would probable be to make USB initialisation conditional on the VBUS being present and wait if it isn't. If the cable is disconnected it also needs to be detected and abort operation because the PLL freezes and so the USB blocks if not handed completely.

Forgive my ignorance here, I've done most of my work in application development, at a much higher level, much more removed from the hardware side of things.  So, the terminology you're using is very new to me.  I believe I understand what you are saying here though, let me know if I'm wrong.  I understand what you just said to mean, that this chip, has a separate phase locked loop, to sync the physical usb clock against, and that PLL needs power on the USB connection to operate.  Correct?

If so, I don't think you would even need any code to shut down that PLL, when the usb connection was removed, for the Teensy anyways, because the Teensy itself is powered from the USB connection, and when you disconnect it, it shuts down.   

you could maybe consider how the PLL's dependency on VBUS presence could be best handled

I assume some of the other boards that your software supports are powered from other sources, and would then continue running, even though the USB was disconnected.  The only thing I can think to do in that scenario, would be something like what you proposed.  i.e. wait until the USB connection was initiated, and then initialize the PLL, and then hook into the USB disconnect, and shutdown the PLL in response to that event.  You could write the code that way, and it should still work for the Teensy I think, it's just that the shutdown code would never get executed.

The Open Source code does have the HS PHY initialisation but it WILL hang if the VBUS is not connected since the PLL will not start (forever loop).

I just checked in the latest HS driver too since there was a bug fix in it for something unrelated (only noticeable in specific situation).

If you can verify its operation (with VBUS connected) on the Teensy 3.6 it would be good

If I am understanding your request correctly, it should mean that all I need to do to test this on the Teensy, is to flip on the high speed USB define, and rebuild, and then try it on the teensy.  I have just finished doing this, and I am still getting the same behavior I was getting before, when I tried the high speed test.  i.e. the LED comes on, and stays solid for almost a full second, then briefly (probably like 100 ms at most) shuts off, then back on again for almost another full second.  Also, the PC will not recognize the device at all.  I don't even see windows trying to enumerate the device, in the message analyzer.  So, it probably is the usb physical clock hung up, like you were saying.

So, I'm not sure how to proceed with the high speed thing, unfortunately, I don't have any kind of a debugger that I could hook to the teensy, to see what is going on inside.  So, I'm totally dependent on the USB interface itself to communicate debug info.

I'm going to work on trying to get an emulated FAT working next, with the full speed mode, so that I can get the MSD requests into my code, and then I'll have to figure out what kind of backside transfer protocol I'm going to use, to transfer the data to the pi, and back.  I'm still willing to do more testing, if I can help you with the HS side of things, so just let me know if there's something you want me to try.  I've also got another idea on how to maybe help the process along, I'll shoot you a private message about it, in a bit :-)

Oh, and one more thing I noticed about your last round of changes, a few posts back, I mentioned a typo that I found in the source code for MSD

I think I found a typo in the new codebase.  In the file usb_cdc_descriptors.h, at line 1112, you currently have:

    (OUT_ENDPOINT | USB_MSB_OUT_ENDPOINT_NUMBER),                       

I believe this should be:

    (OUT_ENDPOINT | USB_MSD_OUT_ENDPOINT_NUMBER),       

As I did the diff to figure out exactly which files I needed to update in my dev version of the code, I saw that this typo was still there.  I've fixed it in my dev codebase, but the USE_USB_CDC option won't compile with that line the way it is.  Just a reminder :-)

Thanks again for all the help so far,

Chris

[mark]

Chris

I had missed a couple of questions:

Oh, and one other question for you.  When I run either of these builds in the simulator, it is reporting a 120kHz clock rate at the top of the screen.  I know the 3.6 supports a 180kHz clock rate, and I was wondering if there is a define somewhere that controls that, or if uTasker maxes out at 120kHz. 

To run at 180MHz rather than 120MHz, enable USE_HIGH_SPEED_RUN_MODE for the board in app_hw_kinetis.h. At this location the speeds of the system clock, bus and flash clocks can also be adjusted if desired.
Beware that 180MHz requires HIGH-SPEED-RUN mode to be used which has some restrictions, such as it is not possible to use internal flash programming. This is why I default to 120MHZ since that uses RUN mode, which doesn't have such restrictions.

I think I found a typo in the new codebase.  In the file usb_cdc_descriptors.h, at line 1112, you currently have:

    (OUT_ENDPOINT | USB_MSB_OUT_ENDPOINT_NUMBER),                       

I believe this should be:

    (OUT_ENDPOINT | USB_MSD_OUT_ENDPOINT_NUMBER),     

This was a spelling mistake, but it didn't cause a problem since the define matched. I had corrected it in the professional version last year, so I just checked it in to the open source one too:
"Correct spelling of USB_MSB_OUT_ENDPOINT_NUMBER to USB_MSD_OUT_ENDPOINT_NUMBER"

I understand what you just said to mean, that this chip, has a separate phase locked loop, to sync the physical usb clock against, and that PLL needs power on the USB connection to operate.  Correct?

If so, I don't think you would even need any code to shut down that PLL, when the usb connection was removed, for the Teensy anyways, because the Teensy itself is powered from the USB connection, and when you disconnect it, it shuts down.

   

The teensy powers via the FS USB interface, which has nothing to do with the HS one. The high speed one is not connected to a connector but only to some holes in the PCB. To use HS I am assuming that one needs to solder an adapter to it.
See the teensy 3.6 circuit diagram and find the pins USB1_DM and USB_DP to see where they are connected to. There is also a solder bridge which connects a regulator (which is marked with a star and so is probably not mounted) to the HS USB's 5V line. I believe that this solder bridge needs to be modified in order to supply the VBUS to the HS USB's regulator (VREG_IN0).

Therefore I believe that the Teensy 3.6 needs always to be powered by the FS USB connection. It then needs an addition USB HS connection 'added' and if this is not supplying the VBUS -> VREG_IN0 signal it is normal that the HS driver will hang since it won't be able to starts its PLL, which is dependent on the VBUS voltage input.

If I am understanding your request correctly, it should mean that all I need to do to test this on the Teensy, is to flip on the high speed USB define, and rebuild, and then try it on the teensy.

 

This is only true if you have done any board modifications necessary and have wired up a connected USB connection to the HS interface.
If you haven't the HS mode won't be able to operate and will hang (if the second USB host's 5V signal is not present).

If you haven't prepared the HW it may be useful to check out whether the Teensy project supports HW use and whether there is a guide to modifying the board for it - maybe there is a small module that can be purchased to solder on to it(?)

I'm going to work on trying to get an emulated FAT working next

There is a guide at http://www.utasker.com/docs/uTasker/uTaskerEmulatedFAT.pdf

as I'm sure I'm going to need some way of debugging my code

Most application layer debugging (once all HW interfaces are operational) can be done in Visual Studio - this is much faster and powerful than HW debugging.
The Teensy 3.6 doesn't have any Jtag debug capability (unless one adds a Jtag adapter to its lines, which is very fiddly). I would invest in a FRDM-K66F since it is quite cheap (not much more that a Teensy 3.6) and has an in-built HW debugger, complete HS USB interface, Ethernet, accelerometer and more and so can be used for all HW debugging. Later you can just switch the define from FRDM_K66F to TEENSY_3_6 to build for the final HW.

Regards

Mark

[creitzel]

Mark,

To run at 180MHz rather than 120MHz, enable USE_HIGH_SPEED_RUN_MODE for the board in app_hw_kinetis.h. At this location the speeds of the system clock, bus and flash clocks can also be adjusted if desired.
Beware that 180MHz requires HIGH-SPEED-RUN mode to be used which has some restrictions, such as it is not possible to use internal flash programming. This is why I default to 120MHZ since that uses RUN mode, which doesn't have such restrictions.

Ah, that makes sense, thanks for the info :-)

The teensy powers via the FS USB interface, which has nothing to do with the HS one. The high speed one is not connected to a connector but only to some holes in the PCB. To use HS I am assuming that one needs to solder an adapter to it.
See the teensy 3.6 circuit diagram and find the pins USB1_DM and USB_DP to see where they are connected to. There is also a solder bridge which connects a regulator (which is marked with a star and so is probably not mounted) to the HS USB's 5V line. I believe that this solder bridge needs to be modified in order to supply the VBUS to the HS USB's regulator (VREG_IN0).

Therefore I believe that the Teensy 3.6 needs always to be powered by the FS USB connection. It then needs an addition USB HS connection 'added' and if this is not supplying the VBUS -> VREG_IN0 signal it is normal that the HS driver will hang since it won't be able to starts its PLL, which is dependent on the VBUS voltage input.

...

This is only true if you have done any board modifications necessary and have wired up a connected USB connection to the HS interface.
If you haven't the HS mode won't be able to operate and will hang (if the second USB host's 5V signal is not present).

If you haven't prepared the HW it may be useful to check out whether the Teensy project supports HW use and whether there is a guide to modifying the board for it - maybe there is a small module that can be purchased to solder on to it(?)

Ah, ok, that makes a lot more sense now.  And I think it completely explains why I'm getting the weird LED behavior when I try to turn on HS USB, since I have no connector in place, it is hanging like you said, and that is producing the behavior I'm seeing.  Again, sorry for my ignorance on these things, but I am learning a great deal as this project progresses.  I will have to check for info on adding a HS USB connector to the teensy, and see what it would involve.  I'm just getting started with the hardware side of things too, so I'm basically a noob at soldering too.  If it sounds relatively straight forward though, I might give it a go.  :-)

I'm not totally sure that I need HS USB, so far my tests I've run with the stereo, have been producing satisfactory/usable performance.  That said, I only have about 20 MP3's on my SD card, and I'm not doing any proxying yet, so it will probably slow down a bit as I progress through the project.

There is a guide at http://www.utasker.com/docs/uTasker/uTaskerEmulatedFAT.pdf

Thanks for that, I will definitely check it out.

Most application layer debugging (once all HW interfaces are operational) can be done in Visual Studio - this is much faster and powerful than HW debugging.

I assume you mean by using the simulator in Visual Studio?  Yeah, I was definitely planning on taking advantage of that.  That said, My code never comes out perfect on the first try, and I anticipate, maybe, having to be able to dump some debug info while the code is running on the teensy.

The Teensy 3.6 doesn't have any Jtag debug capability (unless one adds a Jtag adapter to its lines, which is very fiddly). I would invest in a FRDM-K66F since it is quite cheap (not much more that a Teensy 3.6) and has an in-built HW debugger, complete HS USB interface, Ethernet, accelerometer and more and so can be used for all HW debugging. Later you can just switch the define from FRDM_K66F to TEENSY_3_6 to build for the final HW.

Pretty cool chip there.  If I was planning on building multiple projects on the Teensy, that would definitely be a good investment.  This is basically a 1-off project though, as I'm just building it for myself.  That said, if it turns out that I need HS USB, and can't get it by modifying the teensy for some reason, that board looks like it might work in place of the teensy as well.  (Probably overkill really, since I don't need all the extra hardware, but maybe they have another variant that I could use.)  Good to know that there are some compatible options out there, and I wouldn't have to start all over again, if that happens.

Thanks for the great info.  I'll try to keep ya up to date with the status of things.  Especially if I do end up modifying the teensy for HS USB, and can do the testing you initially asked me for.

Chris

[creitzel]

Mark,

Well, I've spent tonight working my way through the emulated fat tutorial, and poking around in the code that supports it.  As a result, I have another question for you.  :-)

From what I can see in the code, it appears that the emulated drive is entirely setup in memory, as arrays, when the program is initialized, and then those data structures are used to answer the USB MSD requests from the host. 

I've been looking in the code, trying to figure out how, if it's possible, I would go about doing things dynamically.  i.e. ideally I would want to be able to proxy requests for a specific file across my back side interface to the Raspberry Pi, so that I could then lookup the given path on the Pi's USB hard drive, and return the appropriate file. 

I am not seeing a way to implement this within your current framework though, and I am wondering if I am on the wrong path here.  Would it maybe be better for me to try to proxy the actual MSD requests across the back interface, and then attempt to implement those on the Pi side of the fence?  Or is there something I'm missing in your emulated FAT layer, that would allow me to do what I want?  Or maybe I could write a replacement for the emulated FAT layer that would allow me to do what I want? 

If you were trying to implement my project, knowing what you know about your codebase, how would you go about attacking it?

Thanks in advance for any advice you can give me,

Chris

[mark]

Chris

The emulated FAT uses const FAT content but not const data content. The data content can be taken form anywhere - also dynamically.

The FAT itself can't be dynamic because it is read in by the host on USB connection (that is, the host then has a local copy of the FAT and so knows how many files are available, their file objects and such. If this were to dynamically changed the host would not know about it unless the USB were enumerated again to synchronise it.

This means that your files (location, size, names etc.) must be fixed at enumeration. The content can be defined/generated when the host requests it.

The restriction of USB-MSD is that only the host may write. If the device writes (or changes things) after enumeration the host knows nothing about it because this is not something expected with USB-MSD and it can result in corruption if both sides could change things. There is no 'synchronisation' mechanism where the device informs the host that it has changed something apart from doing a complete new enumeration (disconnect/connect).

Regards

Mark

[creitzel]

Mark,

The FAT itself can't be dynamic because it is read in by the host on USB connection (that is, the host then has a local copy of the FAT and so knows how many files are available, their file objects and such. If this were to dynamically changed the host would not know about it unless the USB were enumerated again to synchronise it.

This means that your files (location, size, names etc.) must be fixed at enumeration. The content can be defined/generated when the host requests it.

I was aware of the writing restriction, and I think I wasn't clear in what I want to do before.  I understand the FAT must be static once the USB host has connected to the device, and that basically the host then owns the writing to the device.  What I meant by dynamic is this:

I'm going to be using a USB external hard drive, plugged in to my Raspberry Pi, as a media storage drive.  This drive's contents are read-only and fixed for any given running session of the system.  (The contents on the drive will change from time to time, as I add/remove media files, but this will be done while the drive is disconnected from the car computer system, and is connected to my PC.)  So, while the Pi, and the Teensy are connected to the drive, it is read-only, and the FAT is const. 

The only other consumer of the drive, will be the Raspberry Pi, but it will also interact with it in a strictly read-only fashion (i.e. streaming the same media content via a wifi connection to my kids tablets/phones etc).

The problem I was trying to figure out a design to, within uTasker, is this:  currently your emulated FAT code, has a hard coded array, down in the mass storage code, stored in RAM, with hard coded file names and sizes.  I am going to have to build this FAT structure dynamically, when the device starts, by communicating with the Raspberry Pi, and enumerating the files on the USB drive.  I'm not yet sure if I can do this in RAM, or if I will have to store it on the SD Card, as the FAT will probably be a decent size, because the drive has hundreds of folders, and thousands of files (currently about 30 GB of files, but will expand in the future I'm sure), all using long filenames. 

So, I am trying to determine within your code, what functions/callbacks I have to hook, to pull this off.  From what I've seen so far, I think I'm going to have to replace your const data, with function calls back to my code, that will communicate with the Pi, and return the results.  Maybe I just need an initial function to build the FAT structure from the Pi, and then store it in RAM, or on the SDCard, and then use that to serve the USB.  I'm really not sure yet, I was just asking if you had any pointers on which would be the easiest route to attempt, or how you would attempt to tackle a project like this, given your codes architecture.  :-)

Thanks,

Chris

[mark]

Chris

I haven't used the FAT emulation for some time so took a quick look at it to refresh my memory.
In fact I don't think that it is fixed in any way, as long as the define MAXIMUM_DATA_FILES is set to be adequate for your maximum requirement. mass_storage.c is not responsible for the files, just for managing the interface between the user code and the FAT emulation.

It is the application code that defines how many files there are, how they should be named, their length, etc. and later for passing data to be read from them when the host reads them. The application code is free to do this in any way it wants - the simplest being a cost method which is fixed for the product or else dynamically, based on whatever needs to control it.

In case of the reference it is the application that prepares the file list (see fnPrepareEmulatedFAT() in application.c). This has a fixed set in the reference but if you supply your own version of this routine (which is the idea) you can generate a list based on the present SD card content - you will just need to delay the start the USB task till after the SD card is ready and you have collected the list for insertion.

The functions that you need to support are:

• fnGetDataFile() to inform the mass storage task of the files that it wants to be 'visible'
  uDatacopy() to pass data from these files to the mass storage task when it is feeding the USB-MSD with file content.

These functions may be used as they are (probably possible in most cases) or can be modified to do whatever else they need - like generating random file content or encrypting it, etc.

Therefore I don't in fact think that there are any restrictions; you just need to insert your own file list.

Regards

Mark

P.S. If you haven't done it before, set a break point in fnPrepareEmulatedFAT() to see the application setting a fixed list of files - imagine now how you could insert a dynamic list instead.
Then set a break point in fnGetDataFile() to see how the application informs the mass storage task of the list to be used. See whether that is already adequate once you have your file list ready, or else imagine how you could do this call back in a different way to suite your requirement. This is called for each file, until you terminate the list by returning 0.
Of course, do this in the simulator since it is called on each start.




P.P.S.
I moved this conversion to http://www.utasker.com/forum/index.php?topic=1938.0 since it otherwise mixed two topics.