Developer Terms and Conditions › The Development › Molecular Sensing Models › SCiO model inaccuracy & inconsistency
- This topic has 4 replies, 4 voices, and was last updated 7 years ago by Igor NOVICK.
February 22, 2016 at 9:35 pm #firstname.lastname@example.orgParticipantI wanted to share an observation from my preliminary experience using the SCiO sensor, regarding its potential compatibility to actual apps development.Along with the SCiO sensor and the devkit, an example model for hard cheese is provided, to get the hang of things.I have tested the model to get started. In order to reduce “noise”, I have used the SCiO shade tool to place the SCiO sensor at a stationary position on top of an actual hard cheese. Consumer Physics claims the SCiO is able to detect concentrations exceeding 1%. What puzzles me is the fact that each scan yielded results varying on a wide range of concentrations, reaching 2% difference and more from the actual fat concentration of the hard cheese, at about a quarter of the scans I made. I must emphasis that this variance is a result a the very same input for each scan, as the sensor wasn’t moved at all between scans, and the SCiO shade device prevented external illumination from affecting the scan conditions.How come the results ranged at about 2 times the benchmark concentration for detection?Not being able to detect the correct fat concentration might be a problem of the model, but the fact that each scan gave different results is what worries me more. The sensor should be consistent.Is this an actual limit of the product?I wanted to know whether anyone else have stumbled across this issue and if it is a matter that can be solved by a software update or is this inaccuracy is inherent with the device.Thank you
February 23, 2016 at 8:34 am #2693sakrelaastaParticipant
- This topic was modified 7 years, 1 month ago by email@example.com.
- This topic was modified 7 years, 1 month ago by firstname.lastname@example.org.
as I see it, there are two points here: a. That the result of a scan is not the same as the label of the problem and b. There is no consistency on the measurement of the same sample.
The first is easier:
– Generally the labels of the products give “average values”. That means that the took some samples, measured them and then produced the label. Especially on cheese, I know that the number of samples is quite small, and also the label is printed once a year (or even for more years). So, even if you took the sample to a laboratory, the value probably wont be the same as the label. It is just to give you an idea. This creates also a problem with our sampling. When we collect the samples and give the value that the label says (that probably isn’t a exact value), the model we make isn’t that good. On the other hand, if we collect many many samples, the model will improve (because one label will be higher than the real value, the other will be lower, and they will “correct each other”).
About the inconsistency:
– I have also noticed what you mention (different scans with the scio and the sample not moved), but it was with the spectrum. I scanned a liquid (I think it was milk), and the spectrum was different on the right end. After that I noticed that generally the far right end of the spectrum is not so consistence. I can’t imagine what it may be, but because it is toward the end, I think it is (more or less) ok.
– Now, about the different values that the test gave you… The only thing I can think is the difference of temperature. The collection we have has only one sample at 2oC and then all the other samples 20-24oC. What was the temperature of your sample? I have noticed that the spectrum of the same sample changes in different temperatures. I don’t know if the “Preprocessing Methods (Performs normalization of the signal. This is meant to compensate for changing measurement conditions)” can take that into consideration, but I believe that a correct model should have samples with different temperatures, and the one we are given, doesn’t.
But also, I don’t understand exactly the results of the measurements. I suppose you don’t mean that between the measurements there was a 1% difference (the first was 20% fat and the second was increased by 1%, that means went to 20.2%). Because this is a very small and normal difference. If you mean that the first measurement was 20% and the second was 21%. Again it depends on the first value, if it was 3% and went to 4%, it is a very big deal, because it is an increase of 33%. If from 25% went to 26% I believe it is more acceptable 4% increase (taking into consideration also that the model is not the best given).
I believe that in order to “solve” these questions, we need two things. First, the percentage of confidence for a measurement that the team is working to add to the software. It is necessary for any work. The second is for us to perform tests of the model in lab conditions, with known standards of high purity. When we receive the “SCiO liquid accessory”, I am going to create standards of glucose and BSA protein, and perform detailed measurements.
Also I would like to mention that many laboratory equipment, designed for specific measurements, also perform (automatically) more than one measurements, and then give as a result a mean value. We can not expect from anything a 100% “correct” result.March 2, 2016 at 1:26 pm #2850AyeletKeymaster
We would just want to clarify that the hard cheese collection is only consist of ~40 samples.
This amount of samples is mostly sufficient in order to assess feasibility, however it is NOT sufficient amount in order to create a robust and generic hard cheese application.
This data collection is provided as a preliminary data set which should help users taking the first steps with SCiO and SCiO Lab applications.
As for SCiO accuracy we would like to emphasize: the accuracy depends on the application and on the quality and size of the database. The greater the specificity and the higher number of scans in the database, the more accurate the application will be.
For example, an item containing few components, such as a mixture of water and sugar, will be considered more specific and therefore its application will generally be more accurate than that of juice, which largely contains water and sugar but also various additional ingredients.
Temperature as well as other experimental conditions significantly affect the spectrum. The hard cheese data collection which we provide should be tested by hard cheese samples in room temperature. The dairy application, for example, which will be released soon by us, will support materials at a range of temperatures: 4 degree Celsius to 35 degree Celsius.
Regarding the ‘Confidence level’ value, this feature is currently under development and will be provided in future versions of SCiO Lab application.
Also, in terms of hardware: the performance of the next generation SCiO device was shown to be significantly improved comparing to the current device. It may improve consistency and accuracy issues, among others.
Keep us posted!
The Consumer Physics TeamMarch 14, 2016 at 4:45 pm #2950Igor NOVICKSpectator
[B]THE CHEESE FAT% CONTENT DISCREPANCY.[/B]
FAT% that you see on the cheese pack blistering does USUALY indicate the “milk fat measure in the dry matter”, it is not the case with SCIO actual readings. SCIO does take the ‘cheese milky water” into account as well.
Hence, SCIO always displays towards a greater FAT% as against what you see on the cheese blister pack.
IN fact, SCIO does not “size” the cheese ingredients, but uses the reference data from previous laboratory tests. –SCIO Food Expertise.
To my best awareness, SciO Food Expertise Labs has been measuring FAT content in the Cheese samples wet up to 40% moisture; while the Cheese Manufacturers would done measurements of FAT% with a [b]well-dry product, no moisture.[/b]
Thanks.March 14, 2016 at 5:10 pm #2951Igor NOVICKSpectator
ATTENTION, PLEASE ….
Discrepancy of cheese FAT% may well occur 😥 Do not take the cheese blister FAT% as reference number. It is always calculated for the “dry matter”, zero moisture.
SCIO Labs is known to use “just unpacked” cheese samples up to 40% moisture, while the Diary Farms Labs using well-dry cheese samples, zero moisture, when testing for the FAT abundance.
- This reply was modified 7 years ago by Igor NOVICK.
- You must be logged in to reply to this topic.