RTMP to UMSP

[andreq]

Hi,

I've been trying for the last 5-6h to get an RMTP stream trought the umsp server.

I'm using the latest firmware version from SVN which include rtmpdump, so don't ask me why it's there... it is there.

This is a working rmtpdump line (it's a crapy webtv show from here -_-)

Code: Select all

/usr/bin/rtmpdump -r 'rtmp://medias-flash.tou.tv/ondemand/?auth=daEbmaxdkcBbYaXaMcaalcYdwbyd8cbd_c.-bmyIJk-cOW-
4qqBulJpxAE&aifp=v0001&slist=002/MOV/HR/2010-03-15_ASIMON_0001_hr;002/MOV/MR/2010-03-15_ASIMON_0001_mr;002/MOV/BR
/2010-03-15_ASIMON_0001_br' -y 'mp4:002/MOV/HR/2010-03-15_ASIMON_0001_hr.mov' -q


Here is a part of my TouTV umsp plugin (touTV is a website with Canadian/Quebec, show. They broadcast using rtmp)

Code: Select all

<?php
                                $location = "http://127.0.0.1/umsp/plugins/toutv-proxy.php

                $data = array(
                                       'url'         =>  $location,
                                );
                                $dataString = http_build_query($data, 'pluginvar_');
                        
                        
                                $retMediaItems[] = array (
                                        'id'            => 'umsp://plugins/rev3-animetv?' . $dataString, //don't bother, Ive used the rev3 plugin as a base
                                        'dc:title'      => $title,
                                        'res'           => $location,
                                        'upnp:class'    => 'object.item.videoitem',
                                        'protocolInfo'  => '*:*:*:*',
                                );
?>

And here is the proxy :

Code: Select all

<?php

header("HTTP/1.1 200 OK". "\r\n");
header("Content-Type: video/x-flv");

passthru("/usr/bin/rtmpdump -r 'rtmp://medias-flash.tou.tv/ondemand/?auth=daEbmaxdkcBbYaXaMcaalcYdwbyd8cbd_c.-bmyIJk-cOW-4qqBulJpxAE&aifp=v0001&slist=002/MOV/HR/2010-03-15_ASIMON_0001_hr;002/MOV/MR/2010-03-15_ASIMON_0001_mr;002/MOV/BR/2010-03-15_ASIMON_0001_br' -y 'mp4:002/MOV/HR/2010-03-15_ASIMON_0001_hr.mov' -q");
 
?>

I can't understand why it isn't working...

anyhow, before getting to that, I've tried other method :

rtmpgw : It start a web server that take a http GET request and output the RTMP stream right trought it. For some reason, thw wdtvl doesn't get it... it try, but when the stream is finished, it fail saying that file isnt a supported blablabla.

Actually i've tried a direct rtmpgw... like res: http://127.0.0.1:666/ //666 is the listening port I chose)
without success.

Than I created the proxy, to change the header. It used a socket Open to get to rtmpgw... but still it wasnt playing.

In all my try, I was never able to pass the GET param to rtmpgw, I've been only able to start it with predefined -r and -y param thus 127.0.0.1:666 always downloaded the same stream.

There is something working... as I can see on the rtmpgw console that it is loading the stream.

I can even browse using my computer to WDTVLIVE:666/ and it will download the stream. I've renamed it to .flv and it played nicely in mplayercHC. I've tried it on the wdtvl... it played, but the sound wasn't in sync with the video... but that's another problem.

Something I didn't try as I'm not that good with unix (for now) is transcoding using ffmpeg. I think the latest version support rtmpstream... maybe we could use that as an input and output something that the wdtvl can read.

Sorry for that long post and for all the grammar error... English isn't my maiden language.

[PaulF]

I could not get the HDHR RTMP streams to work either until I used the udpxy method from the IPTV thread to convert to an "HTTP get" stream. I hope someone knows how to do the RTMP streams directly.

[andreq]

I've managed to send some data out from the proxy.

The code look something like that (don't have access to it for now)

Code: Select all

$header("HTTP/1.1 200 OK". "\r\n");
$header("Content-Type: video/x-flv");

$handle = popen("/usr/bin/cat /root/test.flv", "r"); //maybe the path is only /bin/cat... I cant remember

while(!feof($handle)) {
     echo fread($handle , 8192);
}

fclose($file)



Code: Select all

$handle = popen("/usr/bin/rtmpdump -r 'rtmp://..........", "r");

using that, It doesnt work. I think the delay before the rtmp connection is made (nothing is shown with -quiet mode) make the fread think its EOF... thus it quite the loop. I might try without the -q mode. I'll parse the output and start echoing only when the data stream is coming.


I'll try again with rtmpgw. I've found out that my .php file had a BOF and that created lots of headache... as I wasnt able to change the header.

I have a better understanding of the VDR plugin proxy, I'll use that as a base (https://svn.wdlxtv.com/filedetails.php?repname=UMSP&path=/plugins/vdr/vdr-proxy.php)

One thing anoying is that using my "cat file.flv" method shows me that H.264 support on the wdtvl is kinda crap. My FLV file use variable framerate h.264 video and the player can't make is mind around that, thus, it plays at crazy framerate or ultra slow (kinda random) and the audio is out of sync with the crazy video playback. I've seen repport on WD's forum about the same behavior using MKV with variable framerate... let's hope they fix that in the next firmware update.

[andreq]

It's alive !

I'Ve been able to relay my rtmp stream from rtmpgw to php to umsp.

Here is the proxy (lots of junk code for now)

Code: Select all

<?php
/*header("HTTP/1.1 200 OK");
header("Content-Size: 65535");
header("Content-Length: 65535");
header("Content-Type: video/x-flv");
header("Accept-Ranges: bytes");
header("Connection: close");*/


/*set_include_path (get_include_path() . PATH_SEPARATOR .'/usr/share/umsp' . PATH_SEPARATOR .'/var/www/umsp');
include_once('funcs_log.php');*/

// ********************************************************
// Settings, please adjust to your needs

// Logging see log_func.php
$logLevel = L_INFO;

// chunksize and log interval
$chunkSize = 8192*8; // 64K
$logInterval = 10; // 10 secs
$timeLimit = 30; // 30 secs, must be greater than $logInterval



$fp = fsockopen("127.0.0.1", 666, $errno, $errstr, 30);
    if (!$fp) {
        header("HTTP/1.1 404 Not Found");
        echo "$errstr ($errno)<br />\n";
       
        exit;
    }

    $out  = 'GET /?r='. $_GET['r'] . '&y=' .  $_GET['y'] .' HTTP/1.0' ."\r\n";
    $out .= "User-Agent: Wget/1.12\r\n";
    $out .= 'Host: 127.0.0.1 ' /*. $URL['host'] */. "\r\n";
    $out .= 'Cache-Control: no-cache' ."\r\n";
    $out .= "Connection: Close"."\r\n"."\r\n";
    fwrite($fp, $out);

   # Create HTTP headers for WDTV from original headers
   $headerpassed = false;
    while ($headerpassed == false) {
        $line = fgets( $fp);
        list($tag, $value) = explode(": ", $line, 2);

        if (stristr($tag, 'Location')) {
            header('Location: ' . $_SERVER['SERVER_NAME'] . $_SERVER['REQUEST_URI']);
            continue;
        }
      if (trim($line) == "") {
            $headerpassed = true;
         header('Content-Type: video/x-flv');
            //header("Content-Size: 65535");
           // header("Content-Length: 65535");
        }
      header($line);
    }
   
    /*_logInfo ('start actual streaming...');
    _logDebug ('meta data: ', stream_get_meta_data($fp));*/

    global $chunkSize, $logInterval, $timeLimit;
    $fSize = 0;
    $startTime = time();
    $timer = time() + $logInterval;

    while(!feof($fp)) {
        $buf = fread($fp, $chunkSize);
        echo $buf;
        $fSize += strlen($buf);
        if (time() >= $timer) {
            // each call to set_time_limit() will reset the timer, so run indefinitely,
            // but in case of hang, stop script after 30 secs
            set_time_limit($timeLimit);
            $timer = time() + $logInterval;
            //_logInfo (sprintf ('%0.2f MB streamed in %d:%02d minutes',$fSize / (1024*1024), (time() - $startTime) / 60, (time() - $startTime) % 60));
        }
    }

    /* alternative code, maybe less cpu consuming, but also less save and less informative
    // completely switch off timelimit, and hope nothing will go wrong...
    set_time_limit(0);
    while(!feof($fp)) {
        $fSize += fpassthru($fp);
        _logInfo ('read junk of size: ' . $fSize);
        sleep(1);
    }
    */

    //_logInfo ('encountered eof, meta data: ', stream_get_meta_data($fp));

    fclose($fp);
?>

And here is how I call it in my plugin :

Code: Select all


$proxyUrl = 'http://127.0.0.1/umsp/plugins/toutv-proxygw.php';

$query = array(
                                       'r'         =>  urlencode('RTMP STREAM PATH'), //same as -r in rtmpdump
                                       'y'         =>  urlencode('RTMP STREAM PLAYBACK'), //same as -y in rtmpdump
                                );
                        $queryStr = http_build_query($query);
                        $location = $proxyUrl . $queryStr;
                               
            $data = array(
                                       'itemurl'         =>  $location,
                                );
                                $dataString = http_build_query($data, 'pluginvar_');
                        
                        
                                $retMediaItems[] = array (
                                        'id'            => 'umsp://plugins/toutv?' . $dataString,
                                        'dc:title'      => $title,
                                        'res'           => $location,
                                        'upnp:class'    => 'object.item.videoitem',
                                        'protocolInfo'  => 'http-get:*:video/x-flv:*',
                                );


Take note that I have rtmpgw running on port 666 (rtmpgw -g 666)

But guest what... I get the damn variable framerate issue (b-rad said it was double framerate)... so its kinda out of sync with the audio... but it work!

Maybe some other rtmp stream doesnt use variable framerate thus could be watchable.

For now I'll try to parse the html/xml for my Tou.TV. I got a nice python script that someone else created for this website that I'm currently porting. Shouldn't take that long!


EDIt :

Actualy... I just opened the an FLV I grabbed using rtmpdump and here are the info :

Format : Flash Video
File size : 20.6 MiB
Duration : 2mn 27s
Overall bit rate : 1 175 Kbps

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Muxing mode : Container profile=Unknown@3.1
Duration : 2mn 27s
Width : 852 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Audio
Format : AAC
Format/Info : Advanced Audio Codec
Format version : Version 4
Format profile : LC
Format settings, SBR : No
Duration : 2mn 27s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Bit depth : 16 bits


... it's not a variable framerate issue... what is it than... I've read on some other forum people where having the same issue with Revision3 shows... that it was related to the Main@L3.1 profile... that main High@L3.1 would be OK.

I'll try to test some of those tehory soon... But still, I can't change Tou.TV encoding method... I might try to contact them... I did before... there web player is a peice of **** that does support hardware accelerated h264.

Or maybe WD will fix the issue... because the file play perfectly on my computer with MplayerC HC and DXVA

[andreq]

While trying to port my python script to php I hit a bump.

It's been 4 years since I've touch PHP... I'm kinda getting it back bit by bit.

I have a regex in python that look like this :

Code: Select all

pid = re.search('(?<=pid":")\w+', data).group(0)

And here how I translated it to php

Code: Select all

preg_match('#(?<=pid":")\w+#', $episodeHtml, $matches);
$pid = $matches[0]; 


Does it make sense ?

[zoster]

Try

Code: Select all

preg_match('/(?<=pid":")\w+/', $episodeHtml, $matches);
$pid = $matches[0];

instead of the positive lookbehind you could also use:

Code: Select all

preg_match('/pid":"(\w+)/', $episodeHtml, $matches);
$pid = $matches[1];

[parnott100]

How are your efforts going? I may have some ideas for you based upon a similar effort to stream video from a HDHomerun.

1) Using rtpdump
Using the PHP popen or passthru functions will not work. According to the PHP documentation these are blocking functions i.e. they do not return until the spawned process finishes. What I think you need to use is proc_open. It is a little more difficult to set up, but because it is non-blocking, you can use fpassthru for the output from the spawned command while it is still running. It seems to work for me. If you want an example just post.

2) Problems with flv files/streams
Don't know about this but here is a guess. From a UPNP GetPtotocolInfo request, the WDTV Live does not reply that it can handle video/flv streams thru UPNP. So maybe it is making an attempt at decoding the stream and just screwing it up. It could be possible to put the flv into another container that the WDTVLive can handle. I know ffmpeg can do all sorts of magic like this.

[andreq]

For now It's "working" with rtmpgateway. Which is basically the same thing as running rtmpdump from PHP. The only difference is that we need to have rtmpgateway running in background...

The advantage with running rtmpdump from php is that we could use ffmpeg in between the rtmpdump output and the umsp server. That is where my skills are limited.

I can confirm that the FLV playback (for FLVs taken from Tou.tv) isn't working on the WDTV from "any" upnp server. My current plugin does the same exact behavior as If I share the file using Windows 7. It plays with a chaotic frame rate and sounds is out of sync.

if you could help me out with using ffmpeg to put the h264 stream in a mp4... I'd appreciate. I'll try to demux the FLV and remux in mp4 on my PC first just to see if it's the container and not the h264 stream that is causing trouble.

I've tryed re-encoding with ffmpeg on the wdtvl... it was running at 5-6 FPS...

Have you any idea where I should put my plugins file on the wdtvl so they will not be deleted on every reboot ?

Thx

[parnott100]

I am by no means an ffmpeg expert, but you may want to try starting with:

Code: Select all

ffmpeg -i inputfile.flv -vcodec copy -acodec copy -f mp4 outputfile.mp4

This should just copy the video and audio streams into an mp4 container. But you may have to experiment.
Use:

Code: Select all

ffmpeg -formats

to find out what decoders/encoders have been compiled into your version of ffmpeg.

Sorry, I can't help you with where to put the plugin on the WDTVLive. I was trying this on my NAS device.

[andreq]

So thx to anyone who helped my on this I finaly got it working.

Here is the RTMP-Proxy :

Code: Select all

<?php
header('Content-Type: '.urldecode($_GET['content-type']));
header('Content-Disposition: attachment; filename="'.urldecode($_GET['title']).'.'.urldecode($_GET['format']).'"');
header("Accept-Ranges: bytes");
header("Connection: close");

$cmd = "rtmpdump -r '" . urldecode($_GET['r']) .  "' -y '" . urldecode($_GET['y']) .  "' -q | ffmpeg -loglevel quiet -i pipe:0 -vcodec copy -acodec copy -f ". urldecode($_GET['format']) ." pipe:1 | cat";
passthru($cmd);

?>


To use it :

Code: Select all

$proxyUrl = 'http://127.0.0.1/umsp/plugins/rtmp-proxy.php?'; //This path might change

$app = PATH_TO_RTMP_APP; // probably IP.IP.IP.IP:PORT/onDemand/ or something. include the server and port
$path = PATH_TOTHE_RTMP_FILE; //probably mp4:XXXXXXXXXXXX
$format = 'flv'; // file format to transcode to. Most time.. flv to flv works good to fix framerate issue. Can't use Mp4 or similar (g3p)... mkv also didnt work. Let me know if you get it working!
$title = 'movieTitle'; //movie title... get it from your plugin... or juste use a default name
$content-type = '"video/x-flv"'; //What content-type should be used in the header.. see $format

$query = array(
      'r'               =>  urlencode($app),
      'y'               =>  urlencode($path),
      'format'       =>  urlencode($format),
      'title'          =>  urlencode($title),
      'content-type'   =>  urlencode($content-type),
);

$queryStr = http_build_query($query);
$location = $proxyUrl . $queryStr;

$mediaItem = array (
      'id'            => 'umsp://plugins/rtmp-null',
      'dc:title'      => $title,
      'res'           => $location,
      'upnp:class'    => 'object.item.videoitem',
      'protocolInfo'  => 'http-get:*:'.$content-type.':*',
);
      
return $mediaItem;


You can use it for any rtmp stream you want. If you can get it trought rtmpdump -r APP -y PATH... it will work with the proxy.

I might try to add a "direct" _GET params which will bypass ffmpeg transcoding if your rtmp doesnt have any framerate issue... why transcode it ... mine did have issue!

One little issue : Make sure to chmod 666 /dev/random and /dev/urandom b-rad will fix that in next update. There was a bug where apache tryed to get some random (rtmpdump need that) but didnt have access

have fun!

I'll iron out the Tou.tv plugin and post it when I'm done