Wow thanks for all your hard work...
Yes. I was discussing this with a friend of mine who was asking about some steels and I noted that the performance was about 2:1 on cardboard or hemp with the caveat that you had to very carefully make sure the materials were the same. He then asked well what if you didn't? And that was one of the things which prompted me to change the way I did this comparison to put some real numbers into a hypothetical.Bodog wrote:... one or two tests on a given steel are unlikely to prove anything unless the cut material is so similar as to be unrealistic
Yes, it is called conclusion bias.... that people will believe whatever new information arises as long as it reinforces their own currently held beliefs, and that once something is published and it reinforces someone's beliefs, it's extremely hard to suppress that information as inaccurate or otherwise faulty.
Justin Bieber.paladin wrote:...
S... but they're getting ready to introduce "Best Musical Score for a Documentary About Previously Undiscovered Amazonian Tribes" on the Golden Globes and I don't want to miss who wins it.
Yeah, I was trying a few things and that really make the visuals obvious. Unfortunately I can't put error bars on stacked charts so I am going to have to switch to using R or another program.tvenuto wrote:Awesome post, the stacked graphs really drive home the point.
Code: Select all
ANOVA - Two Factor Alpha 0.05 Groups Count Sum Mean Variance Column 1 4.00 30.06 7.52 33.51 Column 2 4.00 71.74 17.94 34.36 Column 3 4.00 31.57 7.89 49.81 P-value F critical Column 4 4.00 25.14 6.29 3.91 0.14 8.63 Within Groups 33.00 3.00 11.00 Row 1 4.00 35.63 8.91 38.32 P-value F critical Row 2 4.00 57.19 14.30 76.13 0.00 1.72 Row 3 4.00 32.98 8.24 14.97 Row 4 4.00 32.72 8.18 73.92 Source of Variation SS df MS F P-value F critical Rows 104.11 3.00 34.70 1.20 0.36 3.86 Columns 349.42 3.00 116.47 4.02 0.05 3.86 Error 260.64 9.00 28.96 Total 714.18 15.00
And that is precisely what I took away from this. And little else.Cliff Stamp wrote:" ... In short, if you are trying to do edge retention comparisons, and you don't heavily control the material cut, then it will take a LOT of work for the biases to randomize out and reveal the nature of the steel."
You're good at summarising... :DDonut wrote:I think from the testing in this thread that I can conclude that M4 is awesome and we still need a Para in M4. :)
It is unfortunate they don't teach it in basic statistics as it is very easy to do with even spreadsheets and allows some very complicated questions to be answered. For example often times here on the forums people will complain "there are too many variables!" when people talk about edge retention, however these types of issues are dealt with with the most basic of statistical knowledge. For example lets say you did a edge retention comparison and you didn't control :timlara wrote:Very, very cool analysis, Cliff! I actually have some project management software that uses monte carlo simulations to help you estimate how long projects are going to take based on your performance on previous projects.
M4 actually had the lowest performance to date in total, but as noted the random error is to high it would take at least 10 runs to even hint that the group above 420 was different individually.Donut wrote:I think from the testing in this thread that I can conclude that M4 is awesome ...
The variability of the cardboard was the variable I was actually measuring, if I constrained it then the measurement would be lost. If you want to look at comparisons where I have constrained the material, then I have done many of them, such as :On Edge wrote:
I am curious as to why more effort was not put in to closely controlling the material cut by each steel and potentially harvesting a more defined glimpse into steel performance ... ?
Thank you.Cliff Stamp wrote:" ... The variability of the cardboard was the variable I was actually measuring, if I constrained it then the measurement would be lost. If you want to look at comparisons where I have constrained the material, then I have done many of them, such as :
Ref : http://www.cliffstamp.com/knives/forum/read.php?3,34787" target="_blank.
Statistics to the rescue.KevinOubre wrote:So, at least according to this data, it seems like anything less than an astronomical amount of cardboard is not really of much value in a test of edge retention unless you somehow got extremely similar groups of cardboard, which is unrealistic as pointed out above.
I may do that next time I do a run. I know absolutely nothing about the data compilation and statistical analysis. Guess I need to add more stuff to the research list.Cliff Stamp wrote:Statistics to the rescue.KevinOubre wrote:So, at least according to this data, it seems like anything less than an astronomical amount of cardboard is not really of much value in a test of edge retention unless you somehow got extremely similar groups of cardboard, which is unrealistic as pointed out above.
In the above what I am doing is taking a knife, taking a particular pile of cardboard and cutting it up. Then another knife gets another pile and these piles can be very different. This difference is systematic meaning the entire pile can be more/less damaging to the edge to cut and thus the edge retention seen is more about the cardboard then the steel.
The problem is that there is a systematic error from one type of cardboard to another, what you need to do is make this a random error because if you do then again it will normalize out. The solution is really simple, it is called using random sampling. What you do is take all the cardboard you have and make a large pile and mix it up. Now when you go to do some cutting you take some pieces at random.
This may seem like you made the problem worse but you have not because that sample, even though it will be made out of a bunch of random cardboard will be very consistent in how it effects an edge from one run to the next. When you get new cardboard you just keep adding it to the main pile. Ideally you make the main pile so large that it is basically infinite and very consistent, have it 10X the size of one run is good, 100X is great.
From time to time you can take the same knife and do runs with it and check the long term drift of your cardboard pile. In this way you can calibrate and correct for any drifting - but to be frank this is starting to get to the point you have to be pretty cereal about cardboard cutting.
In short, it isn't a hard problem - science deals with such all the time and much more complicated. It only takes a little methodology to sort it out. If anyone likes doing the cutting but not dealing with the numbers just send them to me, I will happily run the statistics on them and even correct for sampling drift and similar.