Tuesday, March 15, 2011

Dan Killough Test

I’m going to be adding to this, providing links to screenshots and some interesting observations I made of the data over the next week or two but I wanted to make an initial presentation of the data before any more time went by.
The links will be added here at the top of this post for those few, who don’t fall into a coma after trying to digest my ramblings below.
New Link:  Red Rifle w 12-pin Observations
New Link: Some Rifle/Ammo Characteristics & Measuring Tools

Dan’s Test Methodology as Reported to me:
All testing was done at 50 yards indoors off a Pappas one-piece rest on sturdy concrete benches
We shot 2 rifles, Blue and Red, with both bolts. 
Blue Rifle with 6 bolt had a headspace of .046
Blue Rifle with 12 bolt had a headspace of .044
Red Rifle with 6 bolt had a headspace of .042
Red Rifle with 12 bolt had a headspace of .044
Procedure was as follows:
bolt
-fire 5 fouler shots
-fire 1 complete target in order
-swap to bolt
-fire 3 fouler shots
-fire complete target in order
-clean rifle
-fire 5 fouler shots
bolt stays
-fire complete target
-Swap to bolt
-fire complete target
We did this routine 1.5 times for each rifle.  So 3 targets for each bolt with each rifle.  I wanted to do more, but this took us about 5 hours.  Swapping the bolts around was a real pain.

Firing Pin Strike Depth’s Correlation to Velocity & Accuracy:
There were too many combinations of rifles and bolts along with a large variation in strike depth with each combination to show other than a very weak correlation to velocity.
Trying to make a correlation to accuracy was even more difficult and I gave up.
The best I can do is provide a series of charts that visually depict the velocity characteristics of each combination and a chart showing how strike depths with these specific bolts are changed at a 2:1 ratio by headspace increases.
I do have a question for all of you though. Looking at all these charts I get the impression that good ignition was easy to achieve with these rifles, but I don’t know what the firing pin’s mass is or how much spring force is present with the bolt/pin combinations. I’m curious if the rest of you get the same impression after viewing the charts? Remember, I’m looking at only a possible correlation with velocity and I’m assuming most of you will know there may be a correlation to accuracy that’s independent of ignition alone, because of specific differences in the mass and acceleration of different firing pins and their effects on how the rifle “moves” when this energy is released and impacts the cartridge.
The chronograph data:
75 rounds with each configuration of rifle/bolt for a total of 300 rounds fired.
Red rifle with 12-pin bolt:  Avg 1044 fps and 9.2 fps Std D
Red rifle with 6-pin bolt:    Avg 1047 fps and 9.1 fps Std D
Blue rifle with 12-pin bolt: Avg 1036 fps and 8.4 fps Std D
Blue rifle with 6-pin bolt:   Avg 1035 fps and 7.0 fps Std D

Standard Deviation is a measure of the variability in a set of data and in the case of the Red rifle with 12 bolt above, subtracting 1 Std D from the average velocity = 1034.8 (1044-9.2=1034.8). Adding 1 Std D = 1053.2. This range of velocities (1034.8 to 1053.2) can’t overlap when comparing it with other sets of data if you want to show a significant difference in the results.
In the case of the Red rifle, we have a range of (1034.8 to 1053.2) for the 12-pin and (1037.9 to 1056.1) for the 6-pin. These 2 sets of data overlap to the point of being almost unusual and if you were to shoot 2 tests using only the 12-pin or 6-pin….it wouldn’t be surprising to see a larger difference with the same pin.

This is a simplified explanation that I shouldn’t need to make and there are other factors to be considered when using it, but understanding at least this much is mandatory. Judging by the posts I see on most internet forums, there are great number of shooters who “don’t” understand this and maybe this will be helpful to a few.

The accuracy data below should be viewed in the same manner as the chronograph data and I’ve included several different means of comparing the results, including the ARA score produced if the mathematical center of the grouping is positioned over the center of the 100 ring.

 
6-pin vs 12-pin Summery:
GAP Avg of 3-25 Shot Grps and Grp ES Avg of 15x5 Shot Grps
Red rifle 6-pin bolt:    (0.203” +/-0.027 GAP) (0.251” Avg & 0.082 Std D) 2158 ARA
Red rifle 12-pin bolt:  (0.186” +/-0.028 GAP) (0.246” Avg & 0.109 Std D) 2225 ARA
Blue rifle 6-pin bolt:   (0.185” +/-0.024 GAP) (0.240” Avg & 0.051 Std D) 2233 ARA
Blue rifle 12-pin bolt: (0.200” +/-0.023 GAP) (0.265” Avg & 0.064 Std D) 2167 ARA

The GAP software I use is the most efficient statistical means of measuring the size of a grouping and generating the uncertainties necessary to make comparisons with the fewest number of shots. It’s the most important number to look at and I’ll provide a link with a better explanation at the top of this post later.
In this case it doesn’t matter if you use the GAP statistics or the group ES statistics because there are no significant differences with either means of measurement.

I’m going to state my conclusions very carefully and I want you to pay scrupulous attention to where I place the parenthesis in the following statement:
With “these” specific rifles, “these” specific bolts, “these” specific headspaces, and “this” specific test….there is nothing in the data to suggest either pin is significantly different as regards velocity, uniformity of velocity, or accuracy.
That doesn't mean there isn't a difference, it just means this test didn't show it.

Does this mean I wasted my time? Absolutely Not!!! The testing was handled with more care than most, shot in a tunnel, and if I haven’t made any errors while working with this many numbers, we have a lot of useful information/data that can be built on in the future.

 
Results for each configuration of rifle/bolt and all 12 targets:
Red Rifle w 6 Bolt (Target 1):
            GAP Width 25-Shot Grp: (0.226”) +/-0.047”
            5x5-Shot Grp Avg: (0.287”ES) Min 0.168” Max 0.476” SD 0.116”
            ARA Score: (2075)
            Chronograph Results:
            1042 fps and 8.6 SD
Red Rifle w 6 Bolt (Target 2):
            GAP Width 25-Shot Grp: (0.179”) +/-0.038”
            5x5-Shot Grp Avg: (0.210”ES) Min 0.134” Max 0.257” SD 0.051”
            ARA Score: (2300)
            Chronograph Results:
            1051 fps and 7.6 SD
Red Rifle w 6 Bolt (Target 3):
            GAP Width 25-Shot Grp: (0.205”) +/-0.053”
            5x5-Shot Grp Avg: (0.256”ES) Min 0.192” Max 0.332” SD 0.061”
            ARA Score: (2100)
            Chronograph Results:
            1049 fps and 8.5 SD
Red Rifle w 6 Bolt (All Targets - 75 Shots):
            GAP Width Avg: (0.203”) +/-0.027"
            15x5-Shot Grp Avg: (0.251”ES) Min 0.134” Max 0.476” SD 0.076”
            ARA Score Avg for 3 Targets: (2158)
            Chronograph Results:
            1047 fps and 9.1 SD

Red Rifle w 12 Bolt (Target 1):
            GAP Width 25-Shot Grp: (0.252”) +/-0.066”
            5x5-Shot Grp Avg: (0.354”ES) Min 0.268” Max 0.480” SD 0.091”
            ARA Score: (1975)
            Chronograph Results:
            1041 fps and 12.8 SD
Red Rifle w 12 Bolt (Target 2):
            GAP Width 25-Shot Grp: (0.162”) +/-0.040”
            5x5-Shot Grp Avg: (0.226”ES) Min 0.134” Max 0.305” SD 0.079”
            ARA Score: (2300)
            Chronograph Results:
            1043 fps and 7.2 SD
Red Rifle w 12 Bolt (Target 3):
            GAP Width 25-Shot Grp: (0.145”) +/-0.032”
            5x5-Shot Grp Avg: (0.159”ES) Min 0.102” Max 0.229” SD 0.052”
            ARA Score: (2400)
            Chronograph Results:
            1046 fps and 5.7 SD
Red Rifle w 12 Bolt (All Targets - 75 Shots):
            GAP Width Avg: (0.186”) +/-0.028"
            15x5-Shot Grp Avg: (0.246”ES) Min 0.102” Max 0.480” SD 0.109”
            ARA Score Avg for 3 Targets: (2225)
            Chronograph Results:
            1044 fps and 9.2 SD

Blue Rifle w 6 Bolt (Target 1):
            GAP Width 25-Shot Grp: (0.173”) +/-0.037”
            5x5-Shot Grp Avg: (0.204”ES)-0.172”-0.234”-SD 0.028”
            ARA Score: (2300)
            Chronograph Results:
            1032 fps and 6.9 SD
Blue Rifle w 6 Bolt (Target 2):
            GAP Width 25-Shot Grp: (0.194”) +/-0.042”
            5x5-Shot Grp Avg: (0.272”ES)-0.220”-0.325”-SD 0.047”
            ARA Score: (2100)
            Chronograph Results:
            1038 fps and 6.0 SD
Blue Rifle w 6 Bolt (Target 3):
            GAP Width 25-Shot Grp: (0.189”) +/-0.043”
            5x5-Shot Grp Avg: (0.244”ES)-0.171”-0.320”-SD 0.055”
            ARA Score: (2300)
            Chronograph Results:
            1035 fps and 6.8 SD
Blue Rifle w 6 Bolt (All Targets - 75 Shots):
            GAP Width Avg: (0.185”) +/-0.024"
            15x5-Shot Grp Avg: (0.240”ES) Min 0.171” Max 0.325” SD 0.051”
            ARA Score Avg for 3 Targets: (2233)
            Chronograph Results:
            1035 fps and 7.0 SD


Blue Rifle w 12 Bolt (Target 1):
            GAP Width 25-Shot Grp: (0.199”) +/-0.044”
            5x5-Shot Grp Avg: (0.268”ES) Min 0.211” Max 0.336” SD 0.055”
            ARA Score: (2150)
            Chronograph Results:
            1034 fps and 7.7 SD
Blue Rifle w 12 Bolt (Target 2):
            GAP Width 25-Shot Grp: (0.217”) +/-0.042”
            5x5-Shot Grp Avg: (0.293”ES) Min 0.192” Max 0.368” SD 0.066”
            ARA Score: (2050)
            Chronograph Results:
            1033 fps and 8.7 SD
Blue Rifle w 12 Bolt (Target 3):
            GAP Width 25-Shot Grp: (0.183”) +/-0.033”
            5x5-Shot Grp Avg: (0.234”ES) Min 0.142” Max 0.325” SD 0.069”
            ARA Score: (2300)
            Chronograph Results:
            1040 fps and 7.8 SD
Blue Rifle w 12 Bolt (All Targets - 75 Shots):
            GAP Width Avg: (0.200”) +/-0.023"
            15x5-Shot Grp Avg: (0.265”ES) Min 0.142” Max 0.368” SD 0.063”
            ARA Score Avg for 3 Targets: (2167)
            Chronograph Results:
            1036 fps and 8.4 SD

Some Observations of the Red Rifle with 12-pin.

The 3 25-shot targets fired with the Red rifle and 12-pin are unusual because with an identical configuration, the rifle shot both the most accurate target (#3) and the worst target (#1) of the 12 target total.

Is this the perfect example for illustrating how variable groups can be in rimfire, in even the most controlled settings?
Or, was there an unknown factor which contributed to the anomalous results?

Anomalous is too strong a description, but there is a significant statistical difference between Target #1 and Target #3 and after analyzing hundreds, maybe thousands of large groups….I’ll tell you it’s a very rare occurrence to see significant differences with 25 shot samples. Keep in mind I’m referring to the things we’re most curious about that display small differences in results, and “not” a comparison of a factory 10/22 to a purpose built BR rifle.

Target #1 had a GAP generated “Width” of (0.252” +/-0.066”) and that produces a range of (0.186” to 0.318”) for predicting the true accuracy and making comparisons with other results to show statistical significance.
Target #3 had a “Width” of (0.145” +/-0.032”) and a range of (0.113” to 0.177”).

Because those ranges don’t overlap, it means those results are significantly different.

Here are 2 screenshots showing the GAP analysis of the targets in question:
For those statistically challenged ARA shooters who I’m about to lose because the charts above have shocked your mind into apathy, here are the same targets with the shot locations moved to the center of the 100 ring on the ARA target.
These might be considered a visual and scoring depiction of statistical significance, with a target everyone is familiar with.
Now comes the difficult part. Will there be something Dan remembers about that particular target that might explain it? If there isn’t, can I revisit the data and discover something I overlooked?

Here’s the response I got from Dan when I asked about this target:
Not sure why target 1 was not nearly as accurate.  Owner of the rifle was not pleased with the performance on target 1.  It was the first target of the day, and the rifle had been in the pickup all night so it was cold.  That morning was down into the 30’s.  The temperature was the only variable that I could find that might have caused the decrease in accuracy.

It's certainly revealing and quite a coincidence that it happens to be the first target of the day with a cold rifle that results in this poor target, but I don't normally see quite this severe a degradation of accuracy when I test under the same scenario.

The plots below, which my spreadsheets automatically generate with every target I shoot, may show some contributing factors, but no definite cause for the poor performance on the first target. They are velocity on the x-axis and its correlation to vertical on the y-axis.

Target 1 shows a stronger correlation to vertical with velocity than does target 3, but plots and correlation equations can be deceptive or open to interpretation both visually and mathematically. In this case, if you were to throw out the strange very low velocity shot on the first target and scale both plots with identical ranges for vertical on the left and velocity on the bottom….they would look more alike but still display a difference that should be noted.



If your definition for a rifle being in tune, is its ability to shoot a large spread of ammo speeds without producing vertical on the target, then correlation numbers near zero would be your goal.

Is this rifle in “tune”? I honestly don’t know because I’ve never got a correlation number that will consistently stay near zero and there is a dearth of data from any sources other than my own testing results.

If you use one of the links I’ve posted on this blog to the Border Barrels website, you’ll see some of the research that Dr. Geoffrey Kolbe has done on this same subject. It’s also a great source for a lot of rimfire related matters.

I’ve got one more chart to present which may bring up even more questions I don’t have an answer for. This is a plot of the Std Dev of the ammo across the chrono for each target and its correlation to the GAP calculated accuracy of each target.

The Std Dev of the ammo used for the first target is 12.8 fps and it resulted in a GAP width of 0.252” (worst), but the same ammo and same rifle also produced the lowest Std Dev and smallest group (best).



By the strictest definition of “tune”, this might suggest that neither rifle was in tune but with so many combinations of rifle and bolts along with a few other variables that might skew an analysis….I’m not confident in making any conclusions.

Under “Mission Statement” in this blog, I said I was “clueless”….now you know why. LOL



Landy

Firing Pin Strike Depth

Some Rifle/Ammo Characteristics & Measuring Tools

These are a few questions I asked Dan and the response I received:

How did you measure the depth of the firing pin strikes? I attached pics to demonstrate, we measured the deepest point of the indent as the pin face is not parallel with the case.

Are there any close up photos that might illustrate the range of strike depths and their location on the rim? Maybe 3 images showing shallow, average, and deep pin indentations? I attached the pics and labeled them accordingly.  The ‘10deepVs20deep’ shows a .010” strike next to a .020” strike.  You can tell the difference between the two with the naked eye, but it takes a significant difference in strike depth to see it with the naked eye.
The different shapes of the 12 & 6 pins:

Did the barrels differ in length, groove count, or land and groove dimensions?
Not sure, but I will ask

What chamber was used in these rifles and is there a difference in the engraving length of the bullet after chambering with different bolts? Both are Calfee rifles, and we did not measure the engraving length

Is there a measurement for bolt face to breech face clearance and firing pin protrusion from the bolt face counterbore for each bolt? No

Is the firing pin fall being stopped consistently in any of the rifles by a firing pin stop? Did not verify that.

Was there a tuner used on both rifles and did you or your friend think one or both rifles were in tune? Tuner was used on both rifles, and the blue rifle also used a J&J Long Slide.  The owner feels the rifles are in tune and both rifles were set just as if he would have been competing in a match.