Unifi G3 Flex picture tearing
  • I'm not quite sure how to describe this, but every 10 seconds or so a part of the frame smears / tears on two of my Flex cameras. I have the latest build of SS, and I can see this occurring on both the Mac and the App.
    I have just removed a couple of Hickvision cameras that were quite old, but I must admit didn't do this!
    At the moment I don't have the Protect controller running so these were both in their standalone mode with everything set to max.
    They were set up with the Unifi profile
    Also every now and again the picture is delayed by 5-10 seconds....
    If I monitor them directly in a browser they are perfect.
    I have a third Hickvision bullet camera also running and that is perfectly stable
    Before I start a load of testing, has anyone seen anything similar, or got any suggestions where to start?
    Thanks
  • It's difficult to imaging exactly what you are seeing, so when this happens could you please take a screenshot and email it to us at support@bensoftware.com. A screenshot either from the Mac or the iOS app would be fine.

    A network issue is a common cause of these types of problem. How is your Mac and this camera connected? Ideally they should both be wired into the same Gigabit Ethernet switch, is this the case?
  • I have suffered similar with the Unfi G3. The only fix (sadly for me) was to reduce frame rate and connection speed down to around 1mb. These run fine at full connection speed and frame rate in unifi protect software. I suspect it's something to do with how the RSTP is fed from the camera as it needs to be in stand alone to work in security spy. ? I have tried the preset profile in security spy - and manually yet still the only fix seems connection and frame rate related. In my setup they have dedicated cat6 ethernet yet running them throttled. real shame.
  • They’re connected on Gb switch, and yes lowering the frame rate to 10 seems to have cured it, enough to be useable. I have a G3 and G3 pro coming So will try them soon an see if I get the same.
    Also got a UDM pro coming so will run protect and then try streaming from protect, that may work better?
  • Unfortunately running from udmpro had zero affect for me. still had to be low frame rate and throttled connection speed. I have not a clue why. I wonder if Ben can help?
  • Been trying to catch it but it doesn't happen very often and last only about a second. But usually the bottom third of the picture blanks out or goes fuzzy and it freezes for a few moments.
    I can confirm I've now added UDM Pro and I still see it if I bump up the frame rate, its annoying mostly because it triggers the movement alert
  • I am convinced its how Security Spy looks at the incoming RSTP feed as running on Unifi I can run uptp 30Fps and 3Mbs with zero issues. but on security Spy am in the 10fps and under 1mb just to get stability. Ben can unifi be doing something with the RSTP to affect it in this way? thanks
  • I’ve also noticed high latency using the UniFi cameras. I’m pretty sure it’s SS as when I restart the app it catches up.
    I’ve got three G3 cameras and display the time on the screens and can often see one camera in particular be 40 seconds plus delayed. I’m using a quad core Mac Mini and it doesn’t appear to have high cpu usage. Any suggestions?
  • Ben any ideas what can be causing the loss of sync / need to have low frame rates and connection speed to work? Any pointers would be useful.
  • I'm not sure what could be going wrong here. Issues with delay usually only happen with the Mac's CPU is overloaded and can't process frames as fast as they are arriving.

    @evansgo are all your cameras the "G3 Flex" model?

    @jeff444 do you also have the "Flex" cameras, or some other G3 model?
  • Hi Ben the model is UVC G3 I have checked the CPU and its only 11% on this stream with hardware decoding and H264. It never goes past 15%. Really confusing as other streams using cheap cameras are doing higher frame rates and connection speed with no drop in sync/stuttering etc.
    In addition I have tried running from theRSTP stream in unifi protect and using the camera in standalone mode with the same issues. makes no sense to me at all why this should not work in a stable way? Also tried 2 separate cameras with same results. its weird.

    UVC-G3-Bullet / UVC-G3-AF
    • 1080p Full HD, 30 FPS
    • EFL 3.6 mm, ƒ/1.8
    • Outdoor Weather Resistant
    • 802.3af PoE or 24V Passive PoE
    • Built-in Microphone
    • Wall, Ceiling, or Pole Mount
    • High-Power IR Range Extender
  • Hi Ben, currently I have 2 x G3 flex and 1 x G3 pro. Nothing is looking overloaded, the mac mini is not close to working hard. The delay seems to have got better the longer its all left running and I noticed this exact same thing when I only had one G3 flex in standalone mode, for the first day or so it was awful, but then the lag slowly vanished. Never saw it with the hickvision cameras
  • Thanks for the info. I agree, it's very strange, there is no reason for such problems if the Mac's CPU is not overloaded. We've ordered one of these cameras to test with, and when it arrives I hope we will be able to improve things (if there is anything we can do on our side).
  • Thanks so much I have a few of these and would love to use them to replace my cheap china grey market cameras but at moment they are pretty much too unreliable. Thanks ben!
  • Brilliant thanks, let me know if I can help do any testing, I have 5 cams now so can load it up a bit to test stuff
  • I have noticed over time that although one camera connected direct to poe switch over time it slowly drops out of clock sync. Then it starts to have a loss of over 80%. tried different ports / cables and switches. The same slow degradation. I power reset on the camera resets and drift starts after a few hours again.
  • We have received our UniFi camera and have been testing with it, and we have seen a problem that manifests after 10-30 minutes or so, whereby the camera's data rate drops dramatically, the video is delayed, and there is significant packet loss resulting in corrupt video.

    As far as we can tell this is not something that SecuritySpy is doing - the actual data coming in is compromised at this point, and nothing can be done to decode it properly.

    One solution we have found is to use UDP transport rather than TCP transport. TCP is usually more reliable, but we do come across cameras from time to time that have problems with TCP for whatever reason, and operate much more reliably over UDP. UDP should work well provided you have a fast, reliable, wired network in place (not WiFi).

    So, in the latest 5.2.4b12 beta version of SecuritySpy, we have enabled an "RTSP UDP" Format option, which can be selected under Preferences -> Cameras -> Setup.

    So could you please all try this and report back.
  • OK have tried it but it wont connect. Tried it with a G3 Flex and a G3 Pro, I get timeouts.
    I am connecting via Unifi Protect though and not with the Cams in standalone mode, not sure if thats making the difference.
    The only format I can get to work is RTSP-over-HTTP, RTSP on its own doesn't work either, maybe I have something incorrectly set up?
    IP is the UDM Pro
    HTTP set to 7447
    RTSP set to 554
    USername and password are correct
    Manual Config
    Format is RTSP-overoHTTP
    Request has the designated code in it.
  • Connected fine - power cycled cameras and will run for a day and see the results on UDP.
    Normally the daily report highlights the packet loss so will wait to see what it throws up over the next reporting period. Thanks.
  • Camera 1 with Beta release 12 UDP
    - 105 movie files recorded
    - 0 image files recorded
    - 609 MB of data recorded
    - 29 errors
    - 99.8% camera uptime

    Camera 2 Beta with release 12 UDP
    - 178 movie files recorded
    - 0 image files recorded
    - 1.6 GB of data recorded
    - 34 errors
    - 99.6% camera uptime

    Beta Release 10 and running on TCP

    Camera 1
    - 67 movie files recorded
    - 0 image files recorded
    - 246 MB of data recorded
    - 63 errors
    - 99.7% camera uptime

    Camera 2
    - 251 movie files recorded
    - 0 image files recorded
    - 1.0 GB of data recorded
    - 60 errors
    - 99.9% camera uptime

    - 6 of 6 cameras currently online
    - 5605 movie files recorded in total
    - 0 image files recorded in total
    - 51.7 GB of data recorded in total
    - 99.9% software uptime
    - 12% average CPU usage
    - Normal memory pressure

    A decrease in ERRORS and both have remained in clock sync - will have a real 24 hour result tomorrow

    Looking much better so far!

  • What quality stream are you using in Protect? I just cannot get the stream to start in SS on any settings apart from RTSP over HTTP? I tried a camera restart but no luck
  • OK I found the problem, in case it helps anyone else I had the ports wrong. The RSTP needs to be 7447 in this configuration.
  • Good to hear you found the solution @evansgo. Is it working better with UDP?

    The best way to compare is to use the Dashboard feature (either in SecuritySpy's macOS interface itself or via its web interface) and look at packet loss for these specific cameras when using TCP vs. using UDP. Are you both seeing a significant difference here? We certainly are with our test camera.
  • Looking really good today but I have noticed a new issue :-

    motion capture mode has been disarmed. (Failed to record video frame 5575,818 The key frame interval from the network device is too high, locate and change this setting in the device (may be called I-frame interval / I-frame rate / GOV length / Intra frame period))

    Regards
  • Well I wasn't seeing the packet loss problem myself, but since I installed the beta and found the port issue, its all working really well. I've upped the frame rate back to where I wanted it and everything is looking good and perfectly in sync across four cameras
  • ZERO Errors on my 2 G3 cameras in last 24 hours ! Brilliant.

    30FPS and 6Mb connection perfect.
  • OH just decided to bin the feed from unifi - reconfigured the camera into stand alone mode and running in Manual config in SPY, with unifi cam profile selected . Am unable to get UDP to run on stream 0 (s0 is 1080P) at high connection speeds 6mb (direct cat6e connection) Works on TCP if i login to camera and set connection speed below 2MB Really confused. I would have expected to be able to set higher than this. Was set at 6mb (max selectable) when was running in unifi and streaming out to security spy
    I may be doing something incorrect?
  • Further testing - I can remain error free in standalone mode at 30FPS if i select connection rate at 1.2Mbs and UDP. Any higher and the strange degradation returns.
    Really weird, seeing as the camera works at 6Mb within a unifi CCTV. I suspect the RSTP feed maybe lowered in this config hence always looks ok in a unifi setup
  • Thanks @jeff444 for pointing me to this thread after I posted slightly different issues with a similar setup.

    I'm running 15 UniFi cameras currently, a mix of G3, G3 Micro and G3 Dome, with a single G3 Pro and G3 Flex. They run in managed mode with Protect running on a UCKG2+.

    I started with SS 3.x and three cheapo Foscams, but moved to UniFi after being impressed with their network gear, starting with a couple of cameras and building out. I tried them standalone connected directly to SS on my late 2012 Mac mini with 2.5GHz CPU and 8GB RAM, but at least back then (SS4 and no UniFi profile) it wasn't great and the cameras didn't let me use some functionality in standalone mode.

    Soon after, I hooked them up to a UniFi NVR and Ben helped me set up an RTSP feed from there on port 7447 using RTSP (video and audio). It was rock solid and over time I've upgraded to the latest versions of everything involved.

    I forget the sequence, but I've moved from UniFi NVR and UniFI Video to Protect on the CloudKey+, and SS of course moved to V5, which I upgraded to literally within minutes of seeing it released. The AI motion detection was an awesome improvement.

    Aside the Micros which are WiFi only my cameras are all connected to one of 3 UniFi gigabit switches, except one camera (that also happens to be a Micro) which is in a different building that connects over a wireless PTP bridge of about 115m distance back to the main building. I get about 190Mbps each way over that link.

    I had zero issues until 5.2.2 when I started to get several emails a day about excessive packet loss, pretty much from any camera. It was just bearable enough I could ignore it, but it 100% for sure started immediately after the upgrade to 5.2.2.

    This past Sunday I upgraded to 5.2.3, and immediately I started getting slammed with hundreds of excessive packet loss emails within the first day - almost a round robin around my 15 cameras with something from 1 - 20 minutes between emails. I also started to get spurious alerts of vehicle motion from both G3 Domes in my closed and locked garage, or from the "remote" Micro.

    While I absolutely have nothing but respect for the SS product and I have found Ben extremely patient and helpful this second issue was once more an immediate impact of the SS upgrade - nothing changed about the UniFi setup at all.

    Anyway I just downloaded 5.2.4b12 and I had no interrupted connections nor any issue switching to use UDP, so I'll run that for a period and see what difference it brings, and of course report back here.

    I had turned off email error notifications and will quickly know if part of the issue is resolved having re-enabled them just now. Nothing for a few minutes so far so fingers crossed...
  • An update after around 15 hours is that the issues seem to have been resolved.

    I moved 14 of the cameras over to UDP, leaving one as it was previously (presumably using TCP?). I received on email about excessive packet loss around 30 minutes ago, and it was from the camera that has not ben moved over to UDP.

    I also have had no spurious alerts / motion detections in that period.

    Although early days that is very powerful evidence that the issue is resolved by using UDP as I had most definitely seen multiple issues within any 15 hour period since I updated to 5.2.3.

    I have now moved the G3 Dome that was generating most of the spurious motion detections back to a TCP RTSP stream to see if it reverts to previous behaviour and will report back later.

    Further to the above as I didn't hit the post button apparently, I have had a couple of email alerts and one spurious motion detection from the G3 Dome, way better than it had been over the previous 3 days, but still not as it should be.

    I think therefore that 5.2.4b12 seems less noisy with the cameras running in TCP than 5.2.3 was, but with UDP in place of TCP it seems to be spot on.

    What is odd is that it was an SS update that made a step change in the behaviour, not a UniFi one. Anyway, thanks Ben for your awesome support in assisting the two users above to confirm the fix which seems to have resolved my issue entirely.
  • I will agree that I don't understand where the instability has come from. I have an error rate of 0.5-1.5% annoying - yes. But they do work and remain pretty stable. I have to set the camera to no more than 1.6mb connection (max in camera menu is 6mb).As i say ..they are working at 30FPS HW decoded at approx 20-30%. data rate approx 200kb's. Would love to solve the last issue of upping connection speed to the default 6Mb if at all possible.
  • I received a single email today about excessive packet loss from the G3 Dome that had been the worst culprit of phantom motion events. I'm pretty sure I'd get the odd email like that before 5.2.2 as well, but so infrequent it's not a huge issue.

    I've been running at full HD, 15fps and (I assume default) 3000Kbps. Just for fun I set all cameras to 30fps and 6000Kbps just now to see how that goes.
  • Thanks - I cannot run 30FPS above 1.4Mbs on this release. has me beat.
  • So the update is that I'm seeing no difference having upped the camera settings to 30fps and 6000Kbps - everything is back the way I remember it pre-5.2.2. In other words I'm pretty happy again.

    I'm doing that on the individual camera settings, which are managed by Protect, and then SS picks up an RTSP feed from Protect. There is no way to set fps for that configuration in the SS camera settings
  • Would you try a G3 in standalone mode (removed from Unifi Protect) and try at full throttle (30FPS and 6Mb). Thanks in advance. As i suspect the RSTP feed is manipulated within protect and a much lower rate. I am trying to retire Unifi Protect as they are retiring and switching off ALL web based hosting for CCTV in a couple of months.
  • I'll give a spare G3 a go later and report back, I just need to find a POE injector or a spare POE switch port I can easily access.

    What I have spotted is that at 30fps I'm seeing moving objects smear across the image. Even the seconds part of the overlay corrupts (I use Protect to stamp the image). I backed it down to 15fps at 6000Kbps and that looks stable, but any faster smears a bit, the higher the fps the worse the smearing is.
  • I've just finished setting up a G3 in standalone mode, using the Ubiquiti UniFi profile. That forces RTSP and I've left it in the default (I assume) TCP "RTSP Video and Audio" setting for now.

    The camera settings are se to 30FPR and 6000kbps. It's early days but no error messages yet except when it was rebooting. I'll leave it running as I wanted it set it up to watch my 3D printer anyhow and being in standalone mode for that is fine.

    It's not been used for over 18 months so I'll manually upgrade from 4.4.8 to 4.23.8 now. It's 7 pm here, so I'll report back tomorrow if there are any issues.
  • I switched to UDP as there was a lot of screen corruption - interestingly I didn't notice any before the FW upgrade. Perhaps it's a combo of the newer UniFi FW and the SS update that has triggered the issues.

    I'm seeing an odd graininess develop in the image, gradually worsening over a period of a few seconds then and then clearing. I am not seeing the same corruption of the seconds digit I saw with 30fps and 6000kbps from the cameras on Protect.
  • Thats the exact senario I have - I am going to try and roll back 1 camera to 4.4.8 and see how that affects compared to the latest release.
  • I can confirm rollback to camera firmware 4.4.8 and returning to 6Mb and 30Fps with SSpy set to "RTSP Video and Audio" NO errors recored ! I am shocked but happy. Would love to understand how the streams have been manipulated/corrupted on later releases. As this is a step backwards .....but it works! It would be great if Ben could replicate and shed some light on why?

  • These results are all very interesting, this does suggest that newer camera firmware versions are less stable. This doesn't explain the difference being reported between SecuritySpy v5.2.2 and v5.2.3, though we haven't been able to replicate this here nor find any problems with the RTSP/RTP parser in v5.2.3, so that's a bit of a mystery at this point.

    Any kind of smearing/corruption is most likely the result of lost video packets. When a packet is lost, SecuritySpy has to drop the frame that contains that packet, and with temporarily-compressed video data (like H.264, which is what the UniFi cameras provide), where most frames depend on the previous frames, this can produce noticeable corruption.
  • Still 100% error free for me. Not ideal but its working and can live with it! Happy for all the pointers and especially BEN for taking the effort to purchase and investigate.
  • Update: in some further testing, we found that if data backs up in the network buffers for any reason (i.e. if SecuritySpy doesn't read it fast enough), the camera dumps packets and produces corrupt video. There are far better ways for cameras to gracefully deal with such situations without producing corrupt video data, but unfortunately this kind of thing is something we have seen from time to time. Apparently, from the above reports, their older firmware handled this situation better than the newer firmware.

    So, we have made some further changes in the latest 5.2.4b16 beta version of SecuritySpy that should improve the stability here with the standard RTSP TCP format (increasing network buffer sizes, making sure to read all data from the network as soon as it becomes available etc.). So please try this new update, and switch back from RTSP UDP to the standard RTSP option, and report back.
  • Have updated to 5.2.4b16 and RTSP TCP running 4.4.8.67 unifi firmware - will run overnight then upgrade to latest release firmware 4.26.12.67 and report results
  • First overnight run5.2.4b16 and RTSP TCP running 4.4.8.67 unifi firmware RTSP TCP loss.

    Upgraded all now to 4.23.8.67 (latest public stable release from unifi) will run overnight and update results.
  • Well looks like its 100% error free! Run overnight on latest camera release. Well done Ben. Looks like you have fixed it!. You still get some image tearing as people /vehicles pass, but no where near the point it was. Its worth noting this is running over cat6e rather than wifi.
  • Hi Jeff, many thanks for the quick testing. I've been running our UniFi camera continuously for 2 days now, with 100% uptime and zero packet loss, so for me it's totally fixed. If you are still getting some tearing on moving objects, this could still be a bit of packet loss (in which case you might like to go back to the previous firmware), or it could simply be some compression artefacts, in which case you could try increasing the compression quality/bitrate in the camera's settings. Check the Dashboard feature in SecuritySpy to check for packet loss for this camera over the last couple of days - any level of packet loss can result in corrupt video.
  • Checked as suggested - 100% error free. Yup I would say WELL FIXED! Thanks. The tearing I can live with as 99% time does not appear on the recorded steam.
  • looking good here too - thanks
  • Great to hear that.

    @jeff444 - if it's not packet loss, and doesn't appear on the recorded stream, this may be an artefact of the video decoding. In the Camera Info window, click the header bar (where you see the column names) and enable the "Hardware Video Decode Status" column. You should see a "HW" indicator, showing that the camera is being decoded in hardware. Click the indicator and you can force software decode instead. Does this have any bearing on the problem?
  • Will adjust and report back

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!