Winds of Change — Predictive Faults — Tesla
Tesla recently announced that they plan to use “Stress Sensors” deployed throughout the vehicle in order to predict a failure before it occurs. This of course presents an interesting perspective for those keeping an eye on the horizon. Tesla may be an outlier here since their vehicles are a little different than most but I‘m wondering what your thoughts on this are.
Predictive fault codes could be for physical stress measured by position sensors or current monitor for components. Current monitoring of a Module controlled output could be logged and compared. Peak and continuous operating current showing an increasing trend over time may predict component failure. This will lead to more sensitive electronics requiring consistent thorough electrical testing.
How much will you scrutinize that circuit that tested at .4 ohms?
Hi Scott. General Motors does something similar though not to the same level as Tesla. They have what they call, self diagnosing Proactive alerts, as part of the OnStar system. Proactive alerts are available on select 2015 to current models. Battery, starter motor and fuel delivery systems are monitored based on vehicle data and customers receive in-vehicle messages via OnStar as well e-mail when potential performance degradation is detected. The customer must sign up for the service with OnStar. The data is stored off-board and an algorithm is used to analyze and detect degraded performance. Also for 2018 models the fuel delivery system monitoring is no longer included as part of the system.
Thanks for sharing that info. I was aware of the predictive battery related items that GM was performing but didn't know about the fuel system monitoring. Was that something that the no longer support or is now optional? And do you know what they were doing in regards to fuel? Were they monitoring fuel pump performance or fuel trim issues?
There's some interesting predictive methods used in the aviation industry but for the most part they practice time based replacement. However, I have seen methods using the monitoring/logging of EGT's on small aircraft piston engines in identifying leaking exhaust valves which helps to predict a burnt valve.
Hi Scott, the fuel system monitoring is no longer supported. The algorithm used fuel pressure sensor performance as well as fuel pump resistance to predict performance degradation of the fuel delivery system.
Hi Scott. GM has had Proactive Alerts for some years now, beginning circa 2014 Impalas from recall. Here's a Techlink article on the topic from 2016. Starting And Charging (SAC) and Engine Controls Fuel (ECF) are still listed in SI for 2018 Cruze, but ECF has been discontinued for 2019 on the same vehicle. Models previously supported will continue to be supported, but it is important to note Proactive Alerts always require the owner to enroll through OnStar in order to receive the alerts. It is not an automatic procedure.
Regarding the fuel system, as Allan already alluded, monitoring both fuel pressure sensor for pump performance and circuit resistance covers both any physical issues that include "plumbing" and filtering, plus the electrical circuit. In class we use a variable resistor connected in series in the ground circuit on the 2018 Cruze, to simulate the effects on fuel pump short and long term trims, when increasing and decreasing resistance in the ground. As expected, when adding resistance to the circuit, pump trims shifted to increase through higher Pulse Width Modulation (PWM).
However, when my associate was demonstrating this during the class, he experienced a fuel pump failure when the trims were maxxed out due to high ground circuit resistance. I'd been monitoring and recording the PID changes in GDS 2 during set up and as the maximum ground circuit resistance was achieved the engine had already running poorly because the system could not maintain control.
Whether that was an isolated case or not, we do not know. However, I'd predict that the pump was being overworked at that point and failed as it might if maximum positive pump trims would be observed with a pending pump failure, due to an increase in either physical or electrical resistance. Shortly after, one of the available PIDs was removed from GDS 2, although I don't recall which as that was over a year ago now. I can probably figure that out when we set up the 2019 and compare notes.
Thanks for that Martin, I appreciate the intel. I’ll also try to find that Techlink article and fix your link. It appears that the link got borked in your post somehow.
On the disappearance of that pid, do you suspect that was an accident or other?
Hi Scott. Sorry about the link. I probably posted using IE11 instead of Chrome that night. The link seems to be working fine now, so thanks for fixing it.
As for the disappearance of the "ECF" Proactive Alert PID, I would think that it was intentional. We do notice changes from time to time and may never know why, unless we bump into the person directly responsible for making the change. Perhaps the alerts were simply faulting too many fuel pumps prematurely!
Even back in the Tech 2 days PIDSs or tests would disappear, such a the Service Bay Test for evaporative emission systems, which we used on occasion. That was reinstated shortly afterwards. The "Copy PCMCIA Card" function was removed for obvious reasons of unauthorized card cloning.
The same goes for the MOST Ring diagnostic. The "MOST Communication Enable Circuit" in the list below, no longer exists in the GDS 2 Radio menu.
After watching several of Rich Rebuilds videos on Youtube, it's obvious that there's several failure prone devices on these vehicles. There was one video he posted, where a friend of his re-engineered the switches for the door handles, because they are prone to water intrusion, and wire breakage, because of the design. They made a new switch harness with the same switches that Tesla uses, but they used a more flexible wire, and sealed where the wires enter the switch. There was another video where he replaced the contactors in the battery pack with an updated design, because the early design ones were prone to failure.
As far as using software, and sensors to predict failure in these circuits, I don't see any possible way of predicting when a door handle switch wire will break, but the contactor failure one may be able to be predicted.
It is amazing what can be modeled when there is enough data and math applied. I'm currently facinated by KIA's usage of the Knock sensor to predict crankshaft/rod bearing wear/pending failure (knock). …L P1326.
If only they would advise customers to care for their vehicles properly. If they did, that algo probably would never need to be accessed.
Not just 14-15 2.4l. My wife has a 16 optima with a 2.0 turbo and they've just issued a flash update for the ecm to tighten up the knock detection. Gotta take it in this weekend. After the update, if the engine fails they will replace it for free! Unlimited time and mileage according to the paper in the mail. Maybe mine won't fail, I use the correct oil...
Depends what the "Stress Sensors" are. If it is a bunch of stuff which has to be wired back to a data collection device that can be quite costly to put into every vehicle. This is usually reserved for pre-production models and prototypes. It could very well be a mostly software only "device" which many manufacturers already do today. Getting the information phoned home and who owns it is what is on the table right now!
I think a lot of automotive companies have dabbled in prognostics - using data downloaded from vehicles to predict when a component or system will no longer meets it's intended function. A simple example of that is oil minder strategies that tell the customer when to change oil or looking at charging patterns to predict battery failures. I think our experience is that most customers do not want to service their vehicles based on predictions. They want to fix it only when it's broke (unless it will leave them stranded). On the other hand, fleet managers have been looking at vehicle data for many years to help reduce the cost of maintenance and avoid having equipment out of service. Low battery voltage or high engine coolant temperatures are commonly used to bring vehicles in for service. I think the big thing coming, however, is called "big data". There was a group at Ford that was gathering huge amounts of data from management lease vehicles, then using data analytics to look for trends and patterns. I used to look a data from a few vehicles and it was very time consuming. When you have thousands of vehicles spitting out megabytes of data, you have to use data analytic tools to make use of the data. I was amazed at the things they could predict by looking at large amounts of data, things like traffic patterns and parking patterns based on weather, etc. I think this type of data analysis will be extremely useful for engineering. We calibrate using a handfull of hopefully representative vehicles. With big data and vehicles spitting out data, you could look at OBD monitor results (Mode $06) on every vehicle you built for trends and associations. For example, how much variability is introduced into the catalyst monitor based on ambient temperature, altitude, driving style, etc., etc. If you get data for long enough, you may be able to predict when the catalyst will fail emissions. So in the end, I think it will result in more reliable and robust vehicles, not necessarily an email from Midas to get your muffle fixed before it starts making noise.
Thanks for sharing that with us Paul, I really appreciate it. Yes, it’s easy to see one drowning in data trying to decipher what’s important and what’s not. This is where Artificial Neural Networks (ANN) come into play and they’re not going away. Kinda scary and interesting at the same time.
Scott, where can I get more information about ANN ?
While ANN’s have been around for many years now, there really hasn’t been much talk about it within our environment because of the obvious. Most of the learning for me has been through research although I was privileged recently to attend a presentation recently about ADAS, cameras and how these neural networks are ”visualizing” our environment.
Back in about 2005 I learned that one manufacturer began using a special algorithm to predict cylinder-air across the entire operating range of the engine completely different than what had been practiced in the past (speed density & mass airflow blend, aka dynamic airflow). I recently learned about one manufacturer using an ANN on its non-mass air flow equipped engines and they’ve been doing this for at least a few years now.
So for now, it’s a lot of self study, learn some of the basics around the types of NN’s and how they are being used today. They are all around us an I personally find their uses interesting to say the least. Google is your friend.
Thanks ! Time for me to do some reading and research
After chatting with Scott I decided I'm going to post a series of articles on Artifical Neural Nets/Deep Learning. I hope to keep most of the math out and just focus on past, present, and future applications in the automotive and equipment industries. Stay tuned, first one should be up in a day or two.
HI Scott, Ford had a Neural Network misfire monitor in 2005 on the 6.8L V-10, but we removed it from production that same year. It worked great until it didn't. The key to Neural Networks in the training set. You have to teach it what's good and what's bad. We trained the network on "good" engines with various misfire patterns and the NN learned everything perfectly. We finally had great monitor capability on the V-10! After the system went into production, we experienced what we thought were false misfire indications - we couldn't find anything wrong with the trucks or engines. So the dilemma then becomes - do you retrain the network with these field vehicles and call them good? In a traditional misfire monitor, you can tweak the calibration to desensitize it in certain areas. You can't tweak anything on a neural network, you have to retrain it. We didn't know how to deal with field issues - how do you retrain the network if something unexpected happens in the field? You can't just put out a desensitized calibration until you figure things out. So in the end, we had to drop the NN misfire monitor and go back to conventional crankshaft sensor monitor. I'm sure there will be uses for Neural Nets in vehicles, but it may take a while to get back to OBD.
Thanks for the insight Paul, I had no idea this was happening. Training a machine to think like a human is fascinating to me but I think we're quite a ways away from something as intelligent as a human, especially as it relates to human behavior.
Thinking about self-driving cars, the systems today are likely unable to interpret and process what the lady 2 lanes to your left at 45 degrees will be able to do when the vehicles in front of her rapidly stop and she's unable to take the appropriate action because she's preoccupied with putting on her makeup.
Hello Paul, that is very interesting. I certainly understand the training aspects but what actual engine parameter inputs were used for the monitor? Was it the crankshaft monitor only? Or were there other data elements used to train and make the determination of a misfire? The OBD spec. for that has always been quite challenging. I feel for the heavy duty and diesel guys who are being asked to devise a system for those vehicles.
Hi Robert, the way a neural network works is to take various inputs, weight them using a coefficient and feed them through a transfer function which is then used to feed other transfer functions which results in an output. The network coefficients are learned by training the network. You provide the input data and the desired output to develop the coefficients. The Ford system used acceleration data from the crankshaft position sensor, engine rpm, engine load and cam position as inputs. The output was a "misfire" or "no misfire" call for each cylinder event. The network had 23 nodes and 469 coefficients and needed a separate micro in the PCM to do the calculations at the required rate. Training requires data for misfiring and non-misfiring cylinders including all the sources of variability you get in real life including engine/trans/vehicle variability, road conditions, driving conditions, etc, etc. You can probably image how difficult it is to deal with a new/unknown source of variability that crops up in the field. It was a great science project with outstanding results but putting it into production is always more challenging than people think.
Have you looked at the way Toyota describes the learning nature of their A/C systems? Looks very familiar.
I was looking at data from other makes and found multiple duct temp sensors, sun load, sun elevation and sun azimuth as just some of the PIDs and these were Electronic Manual "Single" zone systems..........
Jim, adaptive strategies are very different from neural networks. Neural networks have no adaptive capabilities - they must be trained. Adaptive strategies are widely used in powertrain control software. I can see Toyota making use of them in A/C control.
Paul, perhaps my imprecise application of language is creating some confusion. I am still learning.
Toyota uses the term neural network and even has a diagram much like the article that Scott linked in the post I replied to. I obviously cannot copy and past Toyota SI here, but for anyone looking for documentation.... Go to the NCF (New Car Features) section of SI for a 2016 Tacoma. (This is just one example but the way Toyota puts out information some items are only found in a model revision year so this one will yield the result) Under Body Electrical, Air Conditioning and System Control they describe Neural Network Control and how it differs from traditional methods of calculating control based on inputs.
The description indicates that the system will take the input layer data and pass it to the intermediate layer where the data is weighted and calculations in the output layer determine the output variables. Toyota seems to describe it as learning responses or output based on data gathered and analyzed over time.
I am by no means a computer engineer and I know and respect your background so I have a question if I may. I understand the concept that with AI there is training that goes on. Teaching a system how to think. What is confusing me is the actual application of that training. Teaching a system every expected parameter is what sets up unexpected results it seems (to this layman). As with students, it is a daunting task to teach every single variable to a "thinking" system and benefits seem to diminish. To put it simply I can give a system the tools it needs to achieve a comfort level in the cab based on inputs and outputs. Sun load will not impact all vehicles the same due to something as simple as body color. In those cases the system will look at inputs over time and learn the desired outputs over time to achieve the task. (that would seem to be a logical goal to my mind anyway ;) )Are you saying that every item in the decision process must be trained in these systems? Or do they actually learn appropriate outputs over time for a specific vehicle based on data gathered?
Thanks for being here. You have been one in the comment stream that I have clicked on for many years.
In my limited research on NN’s, I recently met someone who provided my with a simplified perspective on the basic “types” of training taking place in the hidden layers which are as follows:
- Reinforcement learning - telling it what the goal is and it continues to try and learn the goal
- Supervised - data with expected responses
- Unsupervised learning - give data and tell the program what you’d like know about the data.
This may be overly simplified but if helped me understand this topic a little more.
Hers’s a link to an additional resource that might prove to be helpful as well in understanding NN’s.
Thanks for the discussion. I didn't mean to imply that every single possible input needs to be in a neural network. You only have to have the important ones. You would see that show up in the data as the resulting weighting coefficients would very small for the inputs that didn't matter to the outcome. Misfire is a bit different in that all kinds of things affect crankshaft acceleration, which is the primary input to the monitor. We test on a handful of engines, build a 100,000 of them in production, and hope that they are all the same. The powertrain compliance can change over time as components age, people modify vehicles, etc., etc.
I am surprised that Toyota would go to the expense of implementing a neural network for an HVAC system, although I don't really know what they did. (I can't look at the service info.) As you say, a lot of things affect occupant comfort including sun load, car color, humidity, etc. If you want to improve system performance, I would think you would add sensors that give you the most bang for the buck, e.g. sun load sensor, and have the system adapt as best it can to what the customer wants. A neural network will not compensate for insufficient input data.
As part of the patent phrasing I find it interesting that a reason for the stress sensors is that: "performing regular inspections may be time-consuming and costly"
I think a vast majority of technicians that I speak to do not find merit in the evidence of component decay that we can monitor now, and doubt that a shift to this type of system in other manufacturers vehicles would be utilized to its full potential by the majority of technicians.
For years I've gathered a fairly good "rule of thumb" base-line of PIDs for the indicators of component failure or cause of concern, we have today. I find while speaking to most "line" technicians about its use as a diagnostic tool, its either been dismissed as trivial, or never even been considered before the conversation. So I find it hard to believe that it would be utilized in the future if available.
A few examples of data that is available now that can be clear indications of component failure, or simply evidence for a reason to analyse a vehicle system for an impending problem are:
Fuel Trim %
Fuel Pump Trim %
Battery Charging Duty Cycle Trim %
TB Airflow Compensation %
Oxygen Storage Capacity (Mode 6 Data)
I would like to hear the PIDs others use, and maybe their strategy if it is not a obvious one for using data to see trends towards a component failure or issue.
Also, I may just make a new post altogether as to not hijack this one!
I think the point that Keith brings up is valid. I also believe that Tesla knows this as well. This is one of the many reasons I think they will hold off as long as they can letting any type of diagnostic data out to the publi. I suspect in the years to come if and when it does become available they will make the shops go through rigorous training prior to being given those tools.
That would be an interesting thread. Please do.
From experience over the years I’ve learned to never say never. What we know as being absolutely true today may not be in the futire.
I‘m intrigued by your proposal and look forward in assisting with yiur efforts in any way I can.