parsing BolusWizard records depends on BG units

[ecc1]

A user of a 753 pump is using mg/dL for BG units, so it is possible to change the units on at least some pumps that have MMOL_DEFAULT = True. As a result, his BolusWizard records are being parsed incorrectly, due to

https://github.com/openaps/decocare/blob/master/decocare/records/bolus.py#L141

I think the correct fix is to do a ReadBGUnits call during session initialization, after ReadPumpModel, and to use this dynamic value rather than a static default when decoding records.

issue is descripted here https://gitter.im/nightscout/public?at=57ee9c4fc512d9653a5800e7

today test with
a.) 722
b.) 715
c.) 754

a.) + b). works fine
c.) bug as descripted

see this screenshot

attached treatments for this day by calling

https://myssite.azurewebsites.net/api/v1/treatments.json?find[enteredBy]=/openaps/&find[created_at][$gte]=2016-10-01

treatment_10_01_2016.txt

[christerjensen]

My 715 (mmol) 2.3 FW has the same error. SVG from NS and temp basals are sent to the pump and displayed at Nightscout. I use units conversion, but notice that the bolus wizard from the pump is doing a double conversion somehow. BG entered was 3.7 mmol, which shows as 37, and is plotted as 37/18= 2.1 in Nightscout. https://dl.dropboxusercontent.com/u/24731481/pump.jpg

[christerjensen]

The pump is in mmol - low/high set as 5/6 - pump. pump-history.json converts the low/high incorrectly as well. When I entered 5.1 in boluswizard as SVG it appeares as 51 in pump-hostory.json
{
"unknown_byte[8]": 0,
"_type": "BolusWizard",
"bg": 51,
"_byte[5]": 0,
"unknown_byte[10]": 0,
"_description": "BolusWizard 2016-09-06T14:54:30 head[2], body[13] op[0x5b]",
"timestamp": "2016-09-06T14:54:30+02:00",
"_body": "02900f4632000100000000013c",
"bg_target_high": 60,
"sensitivity": 70,
"carb_ratio": 15,
"food_estimate": 0.1,
"unabsorbed_insulin_total": 0.0,
"correction_estimate": 0.0,
"carb_input": 2,
"_head": "5b33",
"unabsorbed_insulin_count": "??",
"_byte[7]": 0,
"bolus_estimate": 0.1,
"_date": "9e760e0610",
"bg_target_low": 50
},
Low/high is not converted - 5/6 is 50/60 in pump-history.json

[dleclercpro]

The BG and carb units are already contained in the bytes sent back by the Carelink stick. The key is in the second byte of the message body part. After several tests, I found the following piece of code to be working in order to decode BGs, carbs, BG targets and carbs sensitivities (I can't seem to get the insulin sensitivities right though, since no other byte seems to change when their value surpasses 256 mg/dL/U):

# Define an indicator dictionary to decode BG and carb bytes
# <i>: [<BGU>, <CU>, <larger BG>, <larger C>]
indicators = {
    80: ["mg/dL", "g", False, False],
    82: ["mg/dL", "g", True, False],
    84: ["mg/dL", "g", False, True],
    86: ["mg/dL", "g", True, True],
    96: ["mg/dL", "exchanges", False, False],
    98: ["mg/dL", "exchanges", True, False],
    144: ["mmol/L", "g", False, False],
    145: ["mmol/L", "g", True, False],
    148: ["mmol/L", "g", False, True],
    149: ["mmol/L", "g", True, True],
    160: ["mmol/L", "exchanges", False, False],
    161: ["mmol/L", "exchanges", True, False]
}

# Search history for specified record
    for i in range(len(device.history)):

        # Try and find bolus wizard records
        try:

            # Look for code, with which every record should start
            if device.history[i] == code:

                # Define a record running variable
                x = i
        
                # Assign record head
                head = device.history[x:x + headSize]

                # Update running variable
                x += headSize

                # Assign record date
                date = device.history[x:x + dateSize]

                # Update running variable
                x += dateSize

                # Assign record body
                body = device.history[x:x + bodySize]

                # Decode time using date bytes
                time = lib.decodeTime(date)

                # Build datetime object
                time = datetime.datetime(time[0], time[1], time[2],
                                         time[3], time[4], time[5])

                # Proof record year
                if abs(time.year - now.year) > 1:

                    raise ValueError("Record and current year too far " +
                                     "apart!")

                # Format time
                time = lib.formatTime(time)

                # Decode units and sizes of BG and carb entries using 2nd
                # body byte as indicator linked with the previously
                # defined dictionary
                [BGU, CU, largerBG, largerC] = indicators[body[1]]

                # Define rounding multiplicator for BGs and Cs
                if BGU == "mmol/L":
                    mBGU = 1.0

                else:
                    mBGU = 0

                if CU == "exchanges":
                    mCU = 1.0

                else:
                    mCU = 0

                # Define number of bytes to add for larger BGs and Cs
                if largerBG:
                    
                    # Extra number of bytes depends on BG units
                    if BGU == "mmol/L":
                        mBG = 256

                    else:
                        mBG = 512

                else:
                    mBG = 0

                if largerC:
                    mC = 256

                else:
                    mC = 0

                # Decode record
                BG = (head[1] + mBG) / 10 ** mBGU
                C = (body[0] + mC) / 10 ** mCU

                # Not really necessary, but those are correct
                BGTargets = [body[4] / 10 ** mBGU, body[12] / 10 ** mBGU]
                CSF = body[2] / 10 ** mCU

                # Add carbs and times at which they were consumed to their
                # respective vectors only if they have a given value!
                if C:
                    device.carbs.append([C, CU])
                    device.carbTimes.append(time)

                # Give user output
                print "Time: " + time
                print "Response: " + str(head) + ", " + str(body)
                print "BG: " + str(BG) + " " + str(BGU)
                print "Carbs: " + str(C) + " " + str(CU)
                print "BG Targets: " + str(BGTargets) + " " + str(BGU)
                print "CSF: " + str(CSF) + " " + str(CU) + "/U"
                print

        except:
            pass

Hopefully this makes sense! :) Might be useful to know I have only tested this on a 722. I don't know what happens when bolus wizard records are larger, as stated in decocare...

[dleclercpro]

Has anyone still issues with this? @bewest might wanna see this?

[ecc1]

Thanks for working on this. I was able to confirm your findings -- I think it's easier to understand by just saying that the BG units are in bits 7:6 of body[1], and the carb units are in bits 5:4. Both 2-bit fields should hold the value 1 (grams, mg/dL) or 2 (exchanges, mmol); 0 and 3 are illegal.

[ecc1]

Do you have any idea whether BG units are encoded somehow in 0x0A records (CalBGForPH in decocare terminology)? There is a lot of data that we're not using, so I suspect it's in there.

[ecc1]

Never mind, I figured it out. BG units are encoded in bits 6 and 5 of the 4th byte (the same one that has the hour field of the timestamp in the low 5 bits, and bit 9 of the glucose value in the high bit.)

[dleclercpro]

I'm glad you could confirm this. Have you got any idea how to correctly decode ISF for values higher than 256 mg/dL/U? When set past this point, the corresponding byte just jumps back to 0, and then starts going up again, without any other one indicating the highest value for said byte has been reached (at least, I didn't see any yet...).

[ecc1]

I've found the extra bit for insulin sensitivities > 255: it's bit 6 of the previous byte. For example, here's a bolus wizard setup record in which I change from sensitivity 35 to 400 (the max on my 522 pump):

5A 0F 1C C6 11 01 11 15 21 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 78 78 0C 64 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 15 21 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 78 78 0C 64 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 44

You can see the "00 23" in the "before" section', which changes to "40 90" in the "after" section.

When that bit gets set, it breaks my decoding of the sensitivity schedule (since that bit is set in a byte that's used for the start time of that sensitivity and needs to be masked off). It looks like this bug is in decocare too (history.py, line 459)