Closed Captioning via Google Speech Recognition 0.0.8

RatWithACompiler submitted a new resource:

Closed Captioning via Google Speech Recognition - Provides closed captioning via Google Speech Recognition Cloud API

Provides optional closed captioning via Google's Cloud Speech Recognition API and OBS' native caption output which is supported by Twitch's built in player and works on livestreams and in VODs on PC and iOS, no Twitch extension required.
Features:

• Captions only when mic isn't muted or hidden to ensure safety
• Works live and in VODs without Twitch extension
• Completely optional for viewers
• Supports OBS delay

Notes:

• The delay is usually less...

Click to expand...

Read more about this resource...

Thanks for making this!

Is there a way to put the CC into a TEXT in screen, like a GDI+ asset?

There isn't currently no. To use for something like open captions I assume?
Wouldn't be that hard to implement (though I am a bit worried about the whole settings becoming too complicated already), I'll look into that.

Looks cool, but I have quick question.

As I understand then the plugin is using https://cloud.google.com/speech-to-text/ , but that looks to be a paid service so how is it possible to use without user providing some kind of credentials?

@RatWithACompiler on the mac version of your awesome plugin, it would be amazing having the ability to add captions as a source in OBS. When you are not using Twitch, you can't see captions. Tried grabbing the caption preview window but on mac window grabbing is SO choppy and not a real option.

Don't know how hard is to do that though. Thanks !

I tried this, was able to see the captions fine in the OBS in the preview but it wasn't reliable showing up on my player which is video.js based. Some of the text did show up but it was random and not complete. I also tried viewing in by using the stream path in VLC and VLC wasn't able to see a captioning track.

Captioning is amazing! Very accurate, very rapid.

One issue.

Here is a VOD from before I added the captions to my OBS stream (sent to Restream, then Youtube, then of course re-encoded by Youtube into their quality presets).

Here is a VOD of mine from now, from now, after I added the captions.
Here is that same VOD on Twitch.

YouTube does not seem to play nice with these captions while re-encoding video. For some reason it causes massive encoding errors in the final product that look absolutely garish.

EDIT: And notice is still goes nuts every 30 seconds or so in this VOD of a much more low-noise, low-motion game...

Anyone gotten this 5o work with Facebook live?

Consider this a bug report -- not only does it not work with YouTube (expected, this is supposed to be for Twitch only) but -- this captioning service can and will corrupt YouTube VODs.

Example: https://youtu.be/jzXmyjPKmS0?t=12300
Here is the same VOD, from the same restream stream, taken untouched from source off Twitch and then reuploaded to youtube WITHOUT captions: https://youtu.be/yn0vyw0Pe5o?t=507

An excellent plugin, could you add Russian for subtitles or tell me how to do it yourself?

javelin911 said:

Anyone gotten this 5o work with Facebook live?

Click to expand...

I tested this (great job btw), running latest OBS on Windows 10. I was able to see captions in the recorded video using Windows 10 media player. I was NOT able to see captions on Facebook Live when I tested....I know this was for twitch but as the above posted, if this is adding a standard format to the stream, shouldn't the other services pick it up?

When I REMUX the video from MKV to MP4 (using OBS) does the caption remain? I tried viewing it from Dropbox...no luck, I have yet to bring the files over to my Mac to experiment (which seems to handle captions better than windows) or upload a test file to my CDN for testing.

Thank you for this...very promising.

For some reason, I must be installing this wrong. Can you comment because I am not seeing it appear in the list. Using OBS x64, most recent version of OBS. I don't want to pay for speech logger yet, and PubNub isn't quite what i am looking for.

WirelessMike said:

I tested this (great job btw), running latest OBS on Windows 10. I was able to see captions in the recorded video using Windows 10 media player. I was NOT able to see captions on Facebook Live when I tested....I know this was for twitch but as the above posted, if this is adding a standard format to the stream, shouldn't the other services pick it up?

Click to expand...

I also tested this using latest version of OSX OBS and plugin....same results. I also have a streaming CDN that I use for work, and it uses JW Player. I tested a stream from both OSX and Windows 10 setups, pluging was working as I was seeing the captions....JW Player 8 reported "unknown captions" (but it DID detect captions) but did not display, which was the same behavior for both OSX and Windows versions, also, the built in experimental captions on the Windows version of OBS had same results. So I am guess there is something in the formatting of the CC that isn't quite the spec the players are looking for?

Hi, were you able to test MAC version for local recordings? I can see the captions while recording but it's not being saved on the output. Thanks.

WirelessMike said:

I also tested this using latest version of OSX OBS and plugin....same results. I also have a streaming CDN that I use for work, and it uses JW Player. I tested a stream from both OSX and Windows 10 setups, pluging was working as I was seeing the captions....JW Player 8 reported "unknown captions" (but it DID detect captions) but did not display, which was the same behavior for both OSX and Windows versions, also, the built in experimental captions on the Windows version of OBS had same results. So I am guess there is something in the formatting of the CC that isn't quite the spec the players are looking for?

Click to expand...

I also would really appreciate it if this worked on Mac OS. At current, it does not generate a timed text file at all as it does for Windows. I also would really appreciate being able to use the GDI text frames to display the voice to text output, I understand that it is burning the text into the stream which is not ideal, but I would totally use that if/when there is ever a working virtual camera plugin working on the Mac too. I would love a way to display the text without burning, but I imagine that would be difficult. I can see the captioning module working in OBS 25.08 in the preview box, but no way to output onscreen, and no output as a text file, I checked in the locally recorded stream (opened with mkvtoolnix, no subtitle file in there either, it's a lame duck at this point on the Mac, my preferred OS. I tested FB Livestream, Twitch, and YouTube with same results. No output anywhere other than preview window in caption plugin.

Coming from a deaf/hard of hearing guy here. Working with CymaSpace on a mission to make media accessible for the deaf, hard of hearing and others. Loving the progress made thus far on Windows. It's not operable yet on the Mac though, and a lot of Deaf/HOH people are on the Mac ecosystem due to the general better accessibility features on that platform, that, and Apple Watch is a pretty reliable alarm for waking up and receiving notifications from our phones.

By the way. Is there a beta version of 0.0.12 of this plugin available for the Mac? I would happily test and report back for early access if there's a chance that it would be useful at all on the Mac platform.

Hello,
We are willing to pay a developer to implement full support for closed captioning in Youtube and Facebook Live with the OBS-captions-plugin.

We are also looking for a developer to create a Mac-compatible version:

Please contact us if you are interested to tackle this challenge: media(at)cymaspace(dot)org.

@RatWithACompiler is there a way you could log the clean text to a file on the PC? Having this would be incredibly useful in conjunction with scripts. I was digging through the codebase and trying to figure out how I could add in an extra staticly named file output to match the OBS preview dock (like transcript_clean_output.txt) so the file contents match the preview dock window exactly. Is this something you could add easily?? I haven't written cpp in a long time and do not have the tools to properly build the DLL's even if I did figure out how to write to a file.

WirelessMike said:

I also tested this using latest version of OSX OBS and plugin....same results. I also have a streaming CDN that I use for work, and it uses JW Player. I tested a stream from both OSX and Windows 10 setups, pluging was working as I was seeing the captions....JW Player 8 reported "unknown captions" (but it DID detect captions) but did not display, which was the same behavior for both OSX and Windows versions, also, the built in experimental captions on the Windows version of OBS had same results. So I am guess there is something in the formatting of the CC that isn't quite the spec the players are looking for prepaidgiftbalance?

Click to expand...

This page captures your voice transcript and transmits the subtitles into OBS via PubNub. If you accidentally close this page, you can always come back.

RatWithACompiler said:

There isn't currently no. To use for something like open captions I assume?
Wouldn't be that hard to implement (though I am a bit worried about the whole settings becoming too complicated already), I'll look into that.

Click to expand...

Checking if there has been any progress on this. Thanks - either as a live asset, or to TXT file of the current buffer that just replaces itself on update.

Also, can there be an option for it to just be on, not required for streaming or recording - using OBS as compositing tool, and the ability to keep it active would be great.

Very cool.