Skip to content

VTDecoderXPCService CPU usage

13

Comments

  • I’ve had a window open (separate screen on my desktop), showing one camera, all day and the CPU idle has slowly gone down from 80% to 55% with 27 threads.

    Shut down that window and the CPU has gone back to 70% idle. Not the 90% idle it was before I opened the window. The thread count hasn’t dropped back to the baseline 21 either.
  • May need to look into some other elements of the H.264 streams like Bit Rate which is an option on some/many cameras. Will try to remember to check what my cameras are set to tonight and report back. Other elements of streams include bit depth like 8 or 10 - I suspect this isn't an option on most cameras. Audio stream format - I've seen these elements cause issues with playback in different players on multiple platforms - one of them could be causing an issue with VTDecoder (if there is no packet loss that could also explain issues)
  • I've been experiencing occasional high VTDecoderXPCService CPU usage that lasts until I restart SecuritySpy (4.1.6), and it's very easy to reproduce consistently.

    The problem can be triggered simply by viewing captured files via the web interface or the SecuritySpy or Spyglass iOS apps.

    Normally, VTDecoderXPCService on my iMac sits at 10% CPU usage or less while SecuritySpy runs. Viewing captured files once causes it to increase to ~110% and stay there indefinitely. Viewing captured files a second time causes another permanent ~100% increase in CPU usage, and so on.

    Viewing captured files appears to trigger some kind of runaway decoding bug. Previous posts by the developer have claimed that since VTDecoderXPCService is an Apple service the bug must not be in SecuritySpy, but this actually seems to be a case of a SecuritySpy bug triggering excessive unnecessary video decoding, which manifests as high VTDecoderXPCService CPU usage.

    The bug is almost certainly in SecuritySpy, not VTDecoderXPCService.
  • BenBen
    edited July 2017
    Hi @yaypie - it is interesting that you are reporting that this is related to viewing captured video via the web server. As far as I'm aware, no one else has reported this.

    Are you viewing the "HQ" files, or the "LQ" files via the web server? Do you see the problem with either, or both?

    It certainly could be a SecuritySpy bug, but at this point it just doesn't seem that this is the case. We have checked SecuritySpy's code thoroughly for any problems related to this but can't find any. Furthermore, we can't reproduce the problem. It's only a handful of users who are reporting this, indicating it's specific to certain hardware/macOS combinations. The common factor here seems to be macOS 10.12, which indicates this is likely a bug in VideoToolbox (the system library responsible for decoding/encoding video) in this particular system version.

    However I am still open to all possibilities.
  • @Ben VTDecoderXPCService CPU usage increases as soon as I visit the "Captured Files" page by clicking either the "View As List" or "View As Grid" button in the web interface, and then remains high until I restart SecuritySpy. It's not necessary to actually view a video in order to trigger the bug.

    The issue also occurs when viewing the list of captured videos via the SecuritySpy or Spyglass iOS apps, which was reported previously in this thread by @donhoffman: http://www.bensoftware.com/forum/discussion/comment/6004#Comment_6004

    After trying various things to see what might have an effect on my ability to reproduce the issue, I discovered that it occurs consistently when the first video in the list is a particular file generated by my Hikvision DS-2CD2032-I. When I recorded a new video from that camera and it became the first file in the list or when I viewed a list in which the first video was from another camera, the bug stopped occurring. When I deleted the new video, returning the previous one to the first position, it began occurring again.

    The video file that causes the issue has a section at the beginning with dropped frames, but is otherwise unremarkable. This camera frequently (but not always) generates files with sections of dropped frames like this, so it may have been a coincidence that I've been able to consistently reproduce it.

    Here's the file in question, in case it's useful to you: https://www.dropbox.com/s/bi3g5alns9p2req/07-11-2017 20-37-54 M Front door.m4v?dl=0
  • I hear what you're saying Ben but I think I agree with @yaypie on this. Still looks like someone's bug causing a runaway process. Here are some more system usage screenshots from tonight.

    This is what the runaway VTDecoder looks like on my Mac Mini:
    https://www.dropbox.com/s/vhvf5k1kudoihq3/Processes.jpg?dl=0

    And this is the thread from Activity Monitor:
    https://www.dropbox.com/s/580th6yo8chibpu/thread.jpg?dl=0

    On this screenshot from iStat (remote monitoring) you can see the usage creeping up. I was connected to the app on my phone for some points of the evening, although I haven't tested exhaustively. Idle was 0:
    https://www.dropbox.com/s/2khekojzwrt19wj/CPU Load.jpg?dl=0

    Finally the effect: maxed out fans and a cooking CPU die:
    https://www.dropbox.com/s/ryphhodyv86o52n/CPU Temp.jpg?dl=0

    I've turned off all recompression, killed audio streams and carefully matched frame rates to no avail. If anyone has any ideas, I'd be interested. I'm half tempted to write a daily security spy restarter cron job.

    Note that killing VTDecoder works, even if you don't restart SSpy - but it's annoying to have to run a top -U to get the PID from a terminal session if you're administering without using the GUI.

    HELP!
  • edited July 2017
    ...And this is what a VTDecoder kill / restart does:
    https://www.dropbox.com/s/w6h38jadmpqjejj/killed.jpg?dl=0

    For good measure, here's the CPU die temp for the last 3mo. The number of cams and the hardware has been unchanged, apart from some tweaking in the last fortnight to try and protect my hardware. There is a clear correlation with the beta and subsequent new release of Security Spy:
    https://www.dropbox.com/s/nzl2xeinncfw01b/CPU Die 3mo.jpg?dl=0

    @Ben is there anything I can do to help fact-finding?
  • Anything? Anything at all?
    Another day of my CCTV machine cooking itself. Not good.
    If we have a lot longer like this I will need to stop using the app.
    @Ben - please can you provide an update?
    There's really nothing unusual about my setup, so I am mystified by this.
  • VTDecoder is a core OS feature - Apple is responsible.

    as for getting PID - you can do a grep after top - this shows just any VTDecoder processes
    top -l1 | grep 'VTDecoder'

    Download an earlier version if you don't need anything added to the newest releases of SecuritySpy and verify that behaviour goes back to normal (other possible causes - macOS has had updates in the same timeframe with 10.12.5 being released in May, and 10.12.4 a month or so before that)
  • Hi Brian. I'll try a version roll-back rather than halting use of the sw.

    Is there a version of 4.x that does not rely on VTDecoder? I'm not keen on going all the way back to 3.x. Thank you.
  • When you view a list of captured files via the web server, SecuritySpy delivers the small thumbnail images that you can see next to each file. This uses the encoder/decoder service, so this could be related somehow (especially if you click the button to load all thumbnails, which may result in thousands of these images having to be generated). We'll investigate whether there is some problem with this mechanism that could explain this issue.

    To answer your question @camguyfbr - VideoToolbox (which spawns the VTDecoder process) is really the only option for macOS apps doing video encoding/decoding. It is fast and powerful, and maintained well by Apple. It seems unlikely that it would have a bug that would cause this problem, however from all the testing we have done and the information we have about this, a bug in SecuritySpy also seems unlikely at this point. But the bug must reside either in VideoToolbox or SecuritySpy and we're keeping an open mind about this.
  • Thanks for the reply.

    I'm not sure if spindumps are helpful at this stage, but last night at about 0440 something happened and the machine went into toaster mode again. I've pulled the spindump from console.... Let me know if the whole log would help at all.

    I'm at a bit of a loss really - although there are bug-squashing efforts underway with several other video-decoding apps (like Chrome) which point to something going on in VTDecoder in some circumstances.

    Event:
    https://www.dropbox.com/s/guaq0j00s616lwf/0440_cooking.jpg?dl=0

    Spindump:
    https://www.dropbox.com/s/ngdo5bt74sp4ms3/spindump.jpg?dl=0

    I will be very peeved if the life of the machine is curtailed by this constant thermal throttling - it feels like it's bouncing off the redline.
  • sudo killall VTDecoderXPCService

    ...and repeat :-|
  • has this problem been resolved yet? I started having it also since I upgraded to version 4: after a few days CPU use goes to 100% and all I can do then is to close securityspy and open it again or, in some cases, I have to restart my Mac. I just upgraded to version 4.2.
  • As far as I have seen this VTDecoder using increasing levels of cpu usage over the course of a day is the same for all systems apart from my 2.3Ghz Mac mini 2012, i7 quad. It shows no usage of the VTD for the last few months, and I am running everything the same as it was when the VTD was using 300%+
    I still see the issue on all systems I install using 2017 iMacs though. So something weird is going on, not only with this VTD increasing issue, but why does the Mac mini show nothing.
    I set my SS to restart every day using automator calendar event, running 7 cameras 2 x 4mp / 5 x 2mp 24 hr recording + motion to USB drive, all running at 10fps.
    I've just completed another project using a brand new Mac 2017 i5, 3Ghz quad, running 13 cameras, all 2mp, 11 @ 5fps, 2 @ 10fps, and it shows VTD at 50% from the start of SS, and then by 24hrs later, cpu is @ 100%, and VTD @ 300+%. That restarts at 10am every day and sends the cpu usage back to 30-40%, and so it begins again!
  • Ok, I may be going mad here, but has anyone updated to High Sierra and noticed the loss of VTDecoder usage?
    I have just updated the new iMac 2017 model I installed on the last project and the cpu usage dropped from 50% rising to 300+% over a day down to 4% rising to only 17% over 12hrs!!
    WTF is that about?

    Have Apple made a big difference in the way video is sandboxed?
  • I had been considering upgrading my Mini to handle more cams because I was always at 100%, since High Sierra I’m consistently around 40%, and I’ve never seen 100% since upgrading. It’s great.
  • Having recently purchased SS and been playing with it on various different machines, I can report having the same issue on one of my iMacs.

    As part of my setup, I have two identical iMacs: 2009 21.5in models, both running the same ram and HDD. On one, in view only mode, there is no increase in CPU usage as time goes by. On another, with the registered version of SS (4 cam licence) with the same setup as the other usage starts at about 40%, then hits 150% within a couple of hours. Whereupon I have to restart SS.

    I wonder if there are any initial ideas to fix the issue?
  • Oh, this might be interesting. Reading all the above, this comment stuck out...

    "The problem can be triggered simply by viewing captured files via the web interface".

    The only difference between the two machines mentioned about is one can be viewed remotely, this is the machine with the registered version of the app, and I did initially think the CPU loading issues were being instigated only after I'd view captured files remotely. I ruled it out, for no reason that it seemed unlikely, but now reading the above comment, I'm wondering if there is anything in this?
  • Weirdly, after a reboot everything is now running perfectly. So no idea what caused the problem, but for now, it's away.

  • Sadly this issue is back, on the two machines. I'll post a new thread about it,
  • Found this VideoToolbox API talk from Apple WDC, Most of it is over my head. But on topic.
    https://developer.apple.com/videos/play/wwdc2014/513/
  • As an update the issue is back with a vengeance! Both machines are hitting max CPU load within n hour or two. I've tried everything I can think of... Rebooting, turning off http & https web feature (only possible on the one machine as the other is in demo mode), not logging into the web interface, closing the camera view when not in use, closing all other apps, and so on. I'm at a loss.

    On one of the machines before going to bed last night I rebooted and only opened SS, this morning VTDecoderXPCService CPU load was over 700%. Nothing was done with the machine other than SS in the interim.

    My only final thoughts to try is changing from 24hr motion detect segments to hourly. Back when I trailed SS on my main work machine I'm pretty sure this is how I ran it, and on that machine I never saw this issue.

    I'm hoping though someone can offer some help on my other thread with some sort of auto kill script for VTDecoderXPCService.

    I don't think though this is a SS issue as such, but something else that isn't playing ball. I just hope a workaround can be found.
  • Installed latest update und experienced this the first time VTDDec on 280%. Restarting mac hoping this goes away.
  • Reboot did not help. VTD goes up 300% Never got stuff like this from Axis Cameras which they are:

    Error from camera "Kurzwarten rechts", it will be closed. (Failed to decompress incoming video frame 22185,71042 Decompression failed)

    Error from camera "Kurzwarten", it will be closed. (Failed to decompress incoming video frame 22185,71042 Decompression failed)
  • edited January 2018
    "Error from camera "Kurzwarten rechts", it will be closed. (Failed to decompress incoming video frame 22185,71042 Decompression failed)"

    Yes, that's the error I get, afterwards, VTD goes mental.

    In terminal if you kill VTD, SS restarts it within a few secs, then everything seems to work fine again until that error message appears. It can go for hours sometimes, but eventually, it happens.

    I'm trying to work with SS support to somehow kill VTD if/when it's load gets too high, but I'm not getting very far: They're reluctant to help me kill a process that causes SS to stop recording. I do understand this, but at the moment it seems like the lesser of all evils, because if I don't kill VTD then it sooner or later it kills my system!

    Rock? Hard place? Humm!
  • What about downgrading SS this happened after the last update for me first time
  • Ben can we have older V4 Versions to download please?
  • This is a weird one because this is an issue that I have never seen - I run SS 24/7 for months at a time, 4 cameras, SS CPU usage is currently round 10% and VTD is 0%, all cameras are being recorded at 1080p/720p at 30fps.

    I am on a 2013 iMac, High Sierra, with the latest official SS.
  • Its possible to limit cpu usage for a specific app. This will only works with OS X 10.10 or higher. Installs to /usr/local without using sudo. Kinda Safer :)
    VTD is causing issue with other apps since the Sierra update. (Dropdox,Photo Agent,etc).
    This has been a solution for others with the same runaway VTD issue. Not sure about SS. Hack at your own risk! if your not comfortable with Command Line Tools. You will need to bone up...or possibly brick your OS or wait for Apple to patch it. This does not seem to be a issue with El Capitan what i'm running. Im going to wait on the upgrade HS till this gets hashed out.
    Homebrew package manager- https://brew.sh/ CPU Throttle- http://www.willnolan.com/cputhrottle/cputhrottle.html
Sign In or Register to comment.