Finally, a real-time low-latency voice chat model

If you haven't seen it yet, check it out here:

https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice#demo

I tried it fow a few minutes earlier today and another 15 minutes now. I tested and it remembered our chat earlier. It is the first time that I treated AI as a person and felt that I needed to mind my manners and say "thank you" and "good bye" at the end of the conversation.

Honestly, I had more fun chatting with this than some of my ex-girlfriends!

Github here (code not yet dropped):

https://github.com/SesameAILabs/csm

``` Model Sizes: We trained three model sizes, delineated by the backbone and decoder sizes:

Tiny: 1B backbone, 100M decoder Small: 3B backbone, 250M decoder Medium: 8B backbone, 300M decoder Each model was trained with a 2048 sequence length (~2 minutes of audio) over five epochs. ```

The model sizes look friendly to local deployment.