7

I am currently writing the communication framework for a web game, the communications map can be seen below: The code is as follows:

test.php:

<!DOCTYPE html>
<html>
    <head>
        <title> Test </title>
        <script>
            function init()
            {
                var source = new EventSource("massrelay.php");
                source.onmessage = function(event)
                {
                    console.log("massrelay sent: " + event.data);
                    var p = document.createElement("p");
                    var t = document.createTextNode(event.data);
                    p.appendChild(t);
                    document.getElementById("rec").appendChild(p);
                };
            }

            function test()
            {
                var xhr = new XMLHttpRequest();
                xhr.onreadystatechange = function () 
                {
                    if(xhr.readyState === XMLHttpRequest.DONE && xhr.status === 200) 
                    {
                        console.log("reciver responded: " + xhr.responseText);
                    }
                }
                xhr.open("GET", "reciver.php?d=" + document.getElementById("inp").value , true);
                xhr.send();
                console.log("you sent: " + document.getElementById("inp").value);
            }
        </script>
    </head>
    <body>
        <button onclick="init()">Start Test</button> 
        <textarea id="inp"></textarea>
        <button onclick="test()">click me</button>
        <div id="rec"></div>
    </body>
</html>

This takes user input (currently a textbox for testing) and sends it to the receiver, and writes back what the receivers response to the console, i have never received an error from the receiver. it also adds an event listener for the SSE that is sent.

reciver.php:

<?php 
    $data = $_REQUEST["d"];
    (file_put_contents("data.txt", $data)) ? echo $data : echo "error writing";
?>

This as you can see is very simple and only functions to write the data to data.txt before sending back that the write was successful. data.txt is simply the "tube" data is passed through to massrelay.php.

massrelay.php:

<?php
    header('Content-Type: text/event-stream');
    header('Cache-Control: no-cache');
    while(1)
    {
        $data = file_get_contents("data.txt");
        if ($data != "NULL")
        {
            echo "data: " . $data . "\n\n";
            flush();
            file_put_contents("data.txt", "NULL");
        }
    }
?>

massrelay.php checks if there is any data in data.txt and if so will pass it using SSE to anyone with an event listener for it, once it reads the data it will clear the data file.

The entire thing actually works perfectly except for the slight ishue that it can take anywhere from 30 seconds to 10 minutes for massrelay.php to send the data from the data file. For a web game this is completely unacceptable as you need real time action. I was wondering if it was taking so long due to a flaw in my code or if not im thinking hardware (Hosting it myself on a 2006 Dell with a sempron). If anyone sees anything wrong with it please let me know thanks.

Subhanshuja
  • 178
  • 1
  • 2
  • 16
James Oswald
  • 470
  • 2
  • 8
  • 21
  • i'd recommend using some sort of timestamped log function to see which method is taking so much time. – Polyov Jul 30 '16 at 17:10
  • @Polyov that sounds like a good idea, how would i go about that? – James Oswald Jul 30 '16 at 17:12
  • [this SO question shows how to use file logging](http://stackoverflow.com/questions/3531703/how-to-log-errors-and-warnings-into-a-file) – Polyov Jul 30 '16 at 17:17
  • what happens if you put `usleep(1000)` into massrelay.php? i can only guess `while(1) { file_get_contents(); }` is putting stress on your machine – Peter Jul 30 '16 at 17:27
  • also what will happen if you will make a test massreceiver like `while(1) { echo "test"; flush(); sleep(3); }` – Peter Jul 30 '16 at 17:30
  • and last thing your output may be buffered, see `output_buffering` in php.ini i always struggled to force `flush` to work see http://stackoverflow.com/questions/4706525/php-flush-not-working – Peter Jul 30 '16 at 17:30
  • @Peter i actually was considering that, i tried doing `sleep(1);` and ended with the same result. as for `while(1) { echo "test"; flush(); sleep(3); }` that actually runs perfectly fast, about 5-10ms responce time. – James Oswald Jul 30 '16 at 17:32
  • @JamesOswald last idea - can you check what will happen if you'll use native functions `fread` `fclose` `fopen` `fwrite` `flock` instead `file_get_contents` `file_put_contents` because the latter may lock files till end of script execution but i honestly dont remember. – Peter Jul 30 '16 at 17:48
  • @Peter dosn't appear to make a difference, i just changed it and its the same behavior, also i'm quite sad you deleted your answer, i was still reading through that documentation you linked me to T_T – James Oswald Jul 30 '16 at 17:53
  • @JamesOswald I undeleted it only because you asked. My "answer" doesnt solve your problem and I feel like I should delete it. Anyways if you are 100% sure problem is not lying behind output buffering and it works with echo+sleep combination you should look around file lock. Don't use file_put_conents and file_get_contents, if thats not it, I can't help. – Peter Jul 31 '16 at 17:07
  • I have tested a lot of realtime protocols over the net, it seems you want to have it realtime, If you want realtime, use websockets, WebRTC or SocketProgramming for thirdpary softwares, no need for SSE because its not nearly fast enough. – PauAI Apr 11 '17 at 00:22

4 Answers4

6

Three problems I see with your code:

  • No sleep
  • No ob_flush
  • Sessions

Your while() loop is constantly reading the file system. You need to slow it down. I've put a half second sleep in the below; experiment with the largest value for acceptable latency.

PHP has its own output buffers. You use @ob_flush() to flush them (the @ suppresses errors) and flush() to flush the Apache buffers. Both are needed, and the order is important, too.

Finally, PHP sessions lock, so if your clients might be sending session cookies, even if your SSE script does not use the session data, you must close the session before entering the infinite loop.

I've added all three of those changes to your code, below.

<?php
    header('Content-Type: text/event-stream');
    header('Cache-Control: no-cache');
    session_write_close();
    while(1)
    {
        $data = file_get_contents("data.txt");
        if ($data != "NULL")
        {
            echo "data: " . $data . "\n\n";
            @ob_flush();flush();
            file_put_contents("data.txt", "NULL");
        }
        usleep(500000);
    }

BTW, the advice in the other answer about using an in-memory database is good, but the file system overhead is in milliseconds, so it won't explain a "30 second to 10 minute" latency.

Darren Cook
  • 24,365
  • 12
  • 95
  • 193
  • dont forget about "magic" file locking by `file_get_contents` and `file_put_contents`. its unreliable af – Peter Jul 31 '16 at 17:14
  • @Peter Can you provide a good link about this "magic"? Do you just mean the above code is dangerous as it doesn't do any explicit locking, so there is a race condition between the call to `file_get_contents()` and the call to `file_put_contents()` ? – Darren Cook Jul 31 '16 at 21:39
  • P.S. Can I plug my book, "Data Push Apps with HTML5 SSE" ? Even though it is two years old, nothing in SSE has changed, and even the browsers haven't changed much. The examples are mostly in PHP, and things like the `@ob_flush();flush();` idiom is covered in the first chapter, IIRC. – Darren Cook Jul 31 '16 at 21:45
  • @DarrenCook Sure enough i plugged in your revised version of massrelay.php and it worked perfectly. The communication time is almost instantaneous. I'm not entirely sure how it works, but the result almost makes me want to read your book (unfortunately its $3 and not free). still might buy a copy though. – James Oswald Aug 01 '16 at 01:04
  • @JamesOswald The cost of the book is small compared to your time wasted by not knowing what you are doing, or compared to the huge number of hours I spent writing it. Get your company to buy it. If this is a hobby project, consider trying to get your local library to stock it. But please don't post links to illegal downloads :-) – Darren Cook Aug 01 '16 at 08:51
  • if one channel is heavy, try to make two channel. one channel one function to push to client it would make better performance – Subhanshuja Nov 19 '19 at 12:02
5

I don't know that writing to a flat file is the best way to do this. File I/O is going to be your big bottleneck here (reading on top of writing means you'll reach that max really quick). But assuming you want to keep on doing it...

Your application could benefit from a PHP session, to store some data so you're not waiting on I/O. This is where an intermediate software like Memcached or Redis could also help you. What you would do is store the data from reciver.php in your text file AND write it into memory cache (or put it into your session which writes to the memory store). This makes retrieval very quick and reduces file I/O.

I would highly suggest a database for your data tho. MySQL in particular will load commonly accessed data into memory to speed read operations.

Machavity
  • 28,730
  • 25
  • 78
  • 91
  • this should be comment now answer. – Peter Jul 30 '16 at 17:29
  • 2
    @Peter It's a bit too broad for a simple comment. And there's no way to build a code example without building a full working model – Machavity Jul 30 '16 at 17:32
  • 1
    he wants to use flat files and SSE so lets help him with that instead asking him to use memcached reddit and databases. and i think his problem lies in output buffering not flat files – Peter Jul 30 '16 at 17:36
  • I was actually going to try doing a database next, the problem with this is during a game you can have large amounts of data coming in, everyone wants to be moving at the same time, the idea with the text file was that many commands could be written to it before it was read by massrelay, which would pass many commands to be parced by the user for display processing. in theory you could do this with a 1 by 1 MYSQL table but i figured it would be faster with file I/O. – James Oswald Jul 30 '16 at 17:38
  • @Peter I've actually done the flat file thing myself, and while it works on a small scale, it dies a terrible death if you try to scale it up. Hence why a memory cache or DB would make more sense. – Machavity Jul 30 '16 at 17:41
  • @JamesOswald dont use flat file Machavity is 100% right, you will get huge problem with file I/O and locks. But check my answer I believe problem lies on buffering problem. – Peter Jul 30 '16 at 17:42
  • @Machavity I agree 100% but your answer is not a solution to his problem, it should be a comment. – Peter Jul 30 '16 at 17:43
1

EDIT: I deleted this answer as OP is saying that my suggested test 1. (see below) is working fine so my theory about output buffering is wrong. But on other hand he is saying the same code with native functions fread fwrite fclose flock doesnt work, so if buffering and file I/O is not a solution i don't know what is it. I removed my post because I don't thinks it's a valid answer. Let me sum this up:

  • error display is enabled E_ALL
  • flush is working fine
  • OP says he used native file functions properly fopen fread fwrite flock and it doesn't help.

If flush is working, file system is working, I can't do anything but trust OP he is right and give up.

So right know my job here is done, I can't help if I can't try it by myself on OP's system, configuration and code.

I undeleted my answer so OP can have links to docs and other people can see my attempt to make a solution.

MY OLD POST I DELETED


1. Work on test massrelay.php

while(true) {
    echo "test!";
    sleep(1);
} 

so you'll be sure that problem is not file related.

2. Make sure you have error_reporting and display_errors enabled.

I am guessing you get response after 30 seconds because PHP script is being terminated after time limit. if you would have errors enabled you would see error message informing you about that.

3. Make sure you actually flush your output and it's not buffered.

that it can take anywhere from 30 seconds to 10 minutes

You being able to see data after 30 seconds make sense because 30 seconds is default value for max execution time in PHP.

It looks like flush() is not working in your screnario, and you should check output_buffering setting in your php.ini file

Please see this: php flush not working

Documentation:

Community
  • 1
  • 1
Peter
  • 15,758
  • 7
  • 46
  • 76
1

Years ago I experimented with flat-file & also storing data in a DB for communication between multiple concurrent users to a server (this was for a Flash game but the same principals apply).

Flat-file offers worst performance as you will eventually run into read/write access issues.

With DB it will eventually also fall over with too many requests especially if you are hitting the DB thousands of times a second and there's no load balancing in place.

My answer is not solving your current problem but steering you in a different direction. You really gotta look at using a socket server. Maybe look into something like: https://github.com/reactphp/socket

A few issues you may experience with using a socket server is the fact that shared hosts don't allow you to run shell scripts. My solution was to use my home PC for the socket communication and use my domain as a public entry point for the hosted game. Obviously we don't all have static IPs to point our games to so I had to use dyndns and back then it was free: http://dyn.com (there may be some other new services that are now free). With using a home server you will also need to setup your router for port forwarding to send any specific port requests on your IP/router to your LAN server. Make sure you are running firewalls on both the router and the server to protect your other potentially exposed ports.

I know this maybe seems complicated but trust me it's the most optimal solution. If you need any help PM me and I can try guide you through any issues you may experience.

Wancieho
  • 681
  • 1
  • 6
  • 23