DATA REDUCTION FORMULAE TABLE

# Values

Tag #

Description

Output

Units

Method of Computation

101

Dynamic Pressure #1

(20 Hz)

1mb

6.30452E3 * Raw 0.0489

102

Dynamic Pressure #2

(20 Hz)

1mb

5.28371E3 * Raw 1.5768

103

Static Pressure #1

(20 Hz)

1mb

1.5791E2 * Raw + 530.37

104

Static Pressure #2

(20 Hz)

1mb

1.0917E2 * Raw + 691.92

105

Rate of Climb

(20 Hz)

1m/s

5.625E4 * Raw, for Raw > = 0

5.287E4 * Raw, for Raw < 0

106

Rosemount Temp

1deg C

mach2 = 5*((l +dyn_pr/stat_pr)**(2/7)-l)

divisor = 1 + 0.195 * mach2

temp = (1.83105E-3 * Raw + 243.16)/divisor-273.16

107

Reverse Flow Temp

1deg C

divisor = 1 + 0.1594 * mach2

temp = (3.02109E-3 * Raw + 222.06)/divisor-273.16

108

Manifold Pressure

1" Hg

3.1098E-3 * Raw + 0.159275

109

Acceleration

(20 Hz)

1g's

6.25E5 * Raw to 7/14/94

6.25E5 * Raw + 1.0 7/14/94

110

Pitch

(20 Hz)

1deg

3.05175E3 * Raw + 50

111

Roll

(20 Hz)

1deg

3.05175E3 * Raw 50

116

Voltage Regulator

1volts

1.5258789E4 * Raw

118

NCAR true air speed

1m/s

3.96744E3 * Raw

119

PMS End Element 1

1volts

120

PMS End Element 2

1volts

121

Interior Temp (computer)

1deg C

3.05175E2 * Raw

123

High Voltage Current

1mA

1.5258789E-4 * Raw

124

Heater current

1amp

3.05175E3 * Raw

130

Event Code bits

1flags

bit 0 = 1 --> system running

bit 1 = 0 -- > in cloud

bit 2 = 0 -- > foil on

131

GPS warning codes

1flags

11 bit codes

140

FSSP counts

15number

Raw

141

FSSP total counts

1number

Sum of tag 140s (tot_count)

142

FSSP ave diameter

1mm

sum of diams / (tot_count)

*2

143

FSSP concentration

cm³

vol = 0.22275 * tas(volume sampling rate (m³/s))
(.22275 mm² = sampling area)

*2

denom = 1 .55 * activ / 100(adjustment for probe activity)

1#/cm³

conc = tot_count /vol / denom

FSSP total mass

g

mass = sum of counts * volumes

*2

144

FSSP water

1g/m³

water = mass / vol / denom * 1.E6

145

Probe Activity

1%

Raw / 10

147

2DC Total Shadow Or

1#/s

Raw

148

FSSP equivalent diameter

1mm

Ratio of sum of diam **3 to sum of diam**2

*2

149

FSSP equivalent diameter variance

1mm

Consult listing or Appendix B (Data Dictionary)

*2

150

Hail size counts

1#/s per channel

Raw

151

Slow Particle

1#/s

Raw

152

Hail total counts

1number

Sum of tag 150s (tot_counts)

153

Hail ave diameter

1cm

sum of diams / tot_counts

154

Hail concentration

1#/m³

conc = tot_counts / (0. 1 * tas)

0.1 m² = sampling area

(0.1 * tas) = volume sampling rate (m³/s)

Hail total mass

g

mass = S_i count_i * volume_i

• 0.9 (0.9 g/cm³ = hail density)

155

Hail water

1g/m³

water = mass / (0. 1 * tas)

160

Top field mill, low res

(20 Hz)

1kV/m

-2.1117E-2 * Raw + 0.094

*3

161

Bottom field mill, low res

(20 Hz)

1kV/m

-9.915E-2 * Raw + 6.4869

*3

162

Left field mill, low res

(20 Hz)

1kV/m

-9.6722E-2 * Raw 0.9751

*3

163

Right field mill, low res

(20 Hz)

1kV/m

-9.7641E-2 * Raw 2.0759

*3

168

Fifth field mill, low res

(20 Hz)

1kV/m

-2.0911E-2 * Raw + 0.1814

*3

169

Sixth field mill, low res

(20 Hz)

1kV/m

-3.8462E-2 * Raw 0.16678

*3

170

HVPS Total Shadow Or

1counts

Raw

171

HVPS Housekeeping Data

^oC

^oC

^oC

^oC

^oC

V

V

^V

hvps1 = Raw(0)

• 0.001 - 23.1

hvps2 = Raw(1)*0.001 - 23.1

hvps3 = Raw(2)*0.001 - 23.1

hvps4 = Raw(3)*0.001 - 23.1

hvps5 = Raw(4)*0.001 - 23.1

hvps6 = Raw(5)*0.0001 - 0.0

hvps7 = Raw(6)*0.0001 - 0.0

hvps8 = Raw(7)*0.0006 - 0.0

172

GPS latitude

1deg

degree + (minute + hundredths/100)/60

173

GPS longitude

1deg

degree + (minute + hundredths/100)/60

174

GPS groundspeed

1m/s

1852 / 36000 * Raw

175

GPS grnd track angle (mag N)

1deg

Raw / 10

176

GPS magnetic deviation

1deg

Raw / 10 (Raw is 32-bits, not 16)

177

GPS time since solution

1s

(Raw + 6.0) /10

*4

178

GPS altitude

1m

Raw / 10 / .3048 (Raw is 32-bits, not 16)

179

GPS time

UT

Time recorded from T-28 GPS receiver.

185

GPS ROC

1m/s

10ths of ft/min to m/s

186

DMT LW

p = 20 * Raw * 1.5258789E-4 (p = Power)

tlfm = (twk + tk)/2(twk = WireTemp; tk = Air Temp)

Thermal Conductivity

cnd = 5.8E-5 * (398 / (125 + tflm)) * (tflm/273)^1.5
cndw = 5.8E5 * (398/(125 + twk))* (twk/273)^ 1.5

Viscosity

visc = 1.718E4

• (393/(120 + tflm)) * (tflm/273) ^1.5
  

Density

dens = pres / (2870.5 * tflm)(pres = Pressure)

fct = p * l * cnd * (twk tk) (l = Wire Length)

Reynold's Number

re = 100 * dens * tas * d/visc(tas = True Air Speed)

Prandtl Numbers

prf = 0.24 * visc/cnd

prw = 0.24 * vscw/cndw

Dry Air Loss

dryp = 0.26 * re ^ 0.6 * prf ^ 0.37 * (prf/prw) ^ 0.25 * fct/0.239
fact = 1.238E6 * 0.239/((2.0*0.19)*(tas * 100 * (597.3 + 373.16 tk)))

*5

LWC

1g/m³

lwc = (pdryp)

• fact

187

DMT Slave

1V

Raw

190

FSSP gated strobes

1number

Raw

191

FSSP total strobes (div by 10)

1number

Raw

192

FSSP reference voltage

1volts

Raw / 25.5

193

Heading

1deg

(Raw*5.49316E-3 7.0) + 8.5 121.9

205

Altitude

1m

4.43077E4 * (1 (stat_pr/l013.3027)**.190284) (stat_pr = Static Pressure)

206

q_e

1K

Saturation Vapor Pressure

mb

svp = 6.1078 * exp(17.26939 * rft/(tempk35.86))
(tempk = RFT in K)

Saturation Mixing Ratio

kg/kg

smr = svp / (stat_pr svp) * 0.622

K

ts = tempk

• (1000/stat_pr) **0.286

q_e

K

thetae = ts

• exp(597.3*smr)/(0.24*tempk))

207

Saturation mixing ratio

1kg/kg

smr from above

208

Point dz/dt

1m/s

(alt - prev_alt (2 secs previous)) / 2

209

Indicated airspeed

1m/s

c = 1 + dyn_pr / 1013.3027

ias = sqrt(5.79E5*(c**(2/7)l))

210

Updraft (uncorrected)

1m/s

ul = change in alt ((i + l) (i1))/2

u2 = (27 - man_pr) * 92

u3 = (1.94254 * ias - 140) * 17.7

updr = ul + (u2 + u3) * 0.00508

211

Calculated TAS

1m/s

sqrt(rftuc

• mach2 * 401.856/divisor)

212

Updraft correction factor

1m/s

gps_gs₁ * (gps_gs₀ gps_gs₂)/2/9.775 (gps_gs = GPS Groundspeed) 0) current1) 1 sec previous2) 2 secs previous

213

Cooper Updraft

1m/s

updraft + updraft correction factor

214

Kopp Updraft

1m/s

dens = 0.34838 * stat_pr /tempk

ang = pitch * 0.0174533

Kopp = ul + (62.12/.65) * accel * 9.775/(dens * calc_tas)

*6

((0.02028/.65)+ang) * calc_tas

216

Turbulence

1cm^2/3/s

Weighted sum of calculated TAS power spectrum

Static and dynamic pressure values, along with RFTs, are fed into a fast Fourier transform routine. Consult program listing.

217

Air density

1kg/m³

0.34838 * stat_pr / tempk

219

FSSP mixing ratio

1g/kg

FSSP_water / density

220

Hail mixing ratio

1g/kg

hail_water / density

221

RFT uncorrected

1deg C

temp = (3.02109E-3 * Raw + 222.06)-273.16

244

JW equiv water

1g/m³

mass = sum of counts * volumes, where diams > 30

microns are treated as equal to 30

water = mass/vol/denom * 1.E6

260

Ambient vert EF

1kV/m

(tfm - 2 * bfm) / 11.2

261

Plane vert EF

1kV/m

(tfm + 2 * bfm) / 11.2

262

Ambient lateral EF

1kV/m

(lfm rfm) / 44.8

263

Plane lateral EF

1kV/m

(rfm + lfm) / 32.48

264

Ambient vert EF (with roll)

1kV/m

cosr = cos(roll_rad)

sinr = sin(roll_rad)

t264 = t260 * cosr + t262 * sinr

265

Ambient lat EF (with roll)

1kV/m

t265 = -t260 * sinr + t262 * cosr

266

E_xq - Earth

1kV/m

sinpitch = sin(pitch_rad)

cospitch = cos(pitch_rad)

*3

grd_trk_ang_deg = fmod(grd_trk_ang_deg + mag_dev, 360.)

grd_trk_ang_rad = 0.017453 * grd_trk_ang_deg

sinhead = sin(grd_trk_ang_rad)

coshead = cos(grd_trk_ang_rad)

Ex = (fm6 1.1*fm5)/4.5

Ey = (lfm-rfm)/44.5

Ez = ((3.6821 * lfm + rfm) * 0.132 fm5) * 0.1735

cpExsh = cospitch

• Ex * sinhead

Ezcrshsp = Ez

• cosroll * sinhead * sinpitch

Ezchsr = Ez

• coshead * cosroll

Eychcr = Ey

• coshead * cosroll

Eyshspsr = Ey

• sinhead * sinpitch * sinroll

E_xq = cpExsh Ezcrshsp + Ezchsr Eychcr - Eyshspsr

267

E_y--- Earth

1kV/m

Exchcp = Ex * coshead * cospitch

Ezchcrsp = Ez

• coshead * cosroll * sinpitch

*3

Ezshsr = Ez

• sinhead * sinroll

Eycrsh = Ey

• cosroll * sinhead

Eychspsr = Ey

• coshead * sinpitch * sinroll

Ey = Exchcp - Ezchcrsp - Ezshsr + Eycrsh - Eychspsr

268

E_z Earth

1kV/m

Ezcpcr = Ez

• cospitch * cosroll

Exsp = Ex * sinpitch

*3

Eycpsr = Ey

• cospitch * sinroll

Ez = Ezcpcr + Exsp + Eycpsr

269

E_y (267) - E_y (265)

1kV/m

t267 - t265(Difference between old and new methods)

270

E_z (268) - E_z (264)

1kV/m

t268 - t264(Difference between old and new methods)

272

GPS deg lat

1deg

integer portion of tag 172 (t172)

273

GPS min lat

1min

fractional part of t172 * 60

274

GPS deg long

1deg

integer portion of tag 173 (t173)

275

GPS min long

1min

fractional part of t173 * 60

276

GPS true bearing

1deg

mod(t175 + t176 + 360,360)

290

Accelerometer (x)

(20 Hz)

1g's

(Raw * 1.5258789E-4 2.511) / 0.492

291

Accelerometer (y)

(20 Hz)

1g's

(Raw * 1.5258789E-4 2.49) / 0.5

292

Accelerometer (z)

(20 Hz)

1g's

(Raw * 1.5258789E-4 2.513) / 0.503