Beta 4.2.4b9/10 - Video decoding issues
  • Hi all,

    Apologies if beta's feedback should go somewhere else (feel free to move). But based on the thread "FTP missing in High Sierra, any working alternative to upload?" and the availability of a "local" copy allowed me to fix a long standing issue with my SecuritySpy => Hazel => OneDrive and AWS upload.

    This local copy allowed me to remove the scanning of Hazel and files and directories being held open awaiting close etc. Which never really worked a 100%. The copy function now works like a charm. Within seconds the motion triggered captures are off-site.

    However when I started with b9 my D-Link cam DCS gave decode errors, and now with b10 after running for some time both D-Link and Hikvision give decoding errors which were not seen before upgrading from 4.2.3. Now fully aware this is the risk of beta, and as such if it's known just ignore my report :-)

    Happy to debug if/where needed.

    The errors are:
    Error from camera "Voordeur", it will be closed. (Failed to decompress incoming video frame 22185,-8969 Image data decompression failed - bad data)

    Error from camera "Buiten", it will be closed. (Failed to decompress incoming video frame 22185,-8969 Image data decompression failed - bad data)

    Thx,
    Stijn
  • Hi Stijn - thanks for the report. However I can't think of any changes that might cause this problem. As a test, please go back to 4.2.3 for a short time to see if the errors persist. If the errors are also being generated by 4.2.3 then this would indicate some other problem - perhaps with your network (WiFi reception issues etc.).
  • Hi Ben,

    Thanks for your answer, I have reverted to 4.2.3.

    The decoding errors are gone. Please find the logs when grepping for 'Failed to decompress incoming video frame' and some string replacements via sed (see below) to fit within the max forum post length.

    2017/08/28 21:12:10: CamError "Hal", closing. 4.1.7b7,22185,-8969 bad data.
    2017/12/24 17:01:30: CamError "Woonkamer", closing. 4.2.1,22185,-8969 bad data.
    2018/02/16 21:31:29: CamError "Woonkamer", closing. 4.2.3,22185,-8969 bad data.
    2018/02/26 07:03:23: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/02/26 23:21:43: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/02/27 14:01:14: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/02/28 07:05:25: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/02/28 12:07:55: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/02/28 17:19:05: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/01 18:18:16: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/01 19:42:56: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/01 23:37:56: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/02 07:11:08: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/02 17:26:38: CamError "Voordeur", closing. 4.2.4b9,22185,-8969 bad data.
    2018/03/02 23:45:48: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 07:59:38: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 12:26:58: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 13:41:28: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 15:17:48: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 17:27:38: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/03 18:42:10: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 07:37:59: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 07:38:00: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 15:34:50: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 16:55:40: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 16:55:40: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 18:37:30: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 18:37:30: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 20:17:30: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 20:17:30: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/04 23:30:13: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 07:02:53: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 12:09:13: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 16:42:23: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 17:45:23: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 17:45:23: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 21:24:13: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 23:41:54: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/05 23:41:54: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 07:00:54: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 08:10:04: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 08:10:04: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 12:13:54: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 12:13:54: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 13:27:14: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 13:27:14: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 14:52:54: CamError "Voordeur", closing. 4.2.4b10,22185,-8969 bad data.
    2018/03/06 14:52:54: CamError "Buiten", closing. 4.2.4b10,22185,-8969 bad data.

    Now it could of course be this happens when uploading to AWS, as this is lot faster now. Let me see if I can find out if the quicker and other method of copy to disk is actually the culprit.

    Thx once again for your feedback, I'll see what I can find out.

    String replacement command: grep 'Failed to decompress incoming video frame' sspy.txt | sed -e 's/Error from camera/CamError/' -e 's/it will be closed/closing/' -e 's/Failed to decompress incoming video frame. Image data decompression failed - //'

    Stijn
  • Ben & others,

    When re-looking at the above errors, it occurred to me, these are roughly at the same time of motion events.. An motion event makes an ~2 minute recording of 3 cams, and then the beta local copy function triggers for all three.

    As such I think I found a correlation; to verify I copied some random 2Gbyte of data with 4.2.3 to the mounted AWS share no errors occurred. Then I did the same with 4.2.4b10, just again some random data, no errors.

    It seems to be the case that every so often when SecuritySpy does the local copy, and as a result the files are uploaded to AWS triggers this behaviour. In the previous workflow the files where copied on the same disk sequentially, and then synced to OneDrive and AWS.

    So I think the "local" function in the beta is most likely parallel, and possibly better in other aspects :-), and therefor triggers this. Apologies for pointing to SecuritySpy; it's most likely in needing more CPU, disk IO speed or even bandwith.

    Stijn
  • Hi Stijn,

    Thanks for the very detailed report and excellent investigation!

    The local copy function is indeed done very efficiently, in parallel on a background thread, and the fact that this coincides with the errors indicates they are related. There are a few main causes of this "bad data" error: one is bad camera firmware, but I think we can rule this out here. The others are insufficient network bandwidth and insufficient CPU. Both these problems can cause lost bytes in the video stream, leading to decoding errors.

    So check the network bandwidth usage and CPU usage (using Activity Monitor) to see if either or both of these are rising too high at any point around the motion event/copy. I suspect it's going to be the network bandwidth, but if it turns out to be the copy itself, then I can add a throttle on this to slow it down a bit.
  • Hi Ben & Others,

    Long story short lots of trial and identical number of error :-). The SecuritySpy local copy function works, but I kept getting errors. So I reverted to Hazel to do the copy job.

    Stijn

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!