zen quaker

A self quanitified zen quaker statistical programmer stumbles through a blog

Tag: data

Habit XI

The Chess tactics problems habit is still chugging along after last night. It’s now up to 12 days, 85 minutes, and 146 chess problems. However I’ve been rethinking the long term plan for further habits. The problem is that in the old plan the third habit was changing my sleep patterns. Changing your sleep patterns is something the teachers of the Habit Course consider a hard habit to change. They recommend that you make a few easy habit changes first before tackling the hard ones. Of course, I’ve already conquered the hard habits of quitting smoking and quitting drugs. But that wasn’t using their method, so I’m going to try and stick (generally) to their program.

Read the rest of this entry »


Fumble I

Today I made the first use of my expanded tasks in Task Coach, where I track what I do at work. It forced me to change some of my behaviors. I often switch back and forth between different tasks. I’ll be working on task A, answer a fresh email on task B, go back to task A, have an idea and run off to talk to someone about task C. Now when I get that email about task B I have to ask myself, do I really need to deal with this right now? Sometimes I do, sometimes I do. But instead of mentally shifting gears I also have to physically shift gears by going over to Task Coach and shifting what it is tracking. I think it’s bringing more focus to my work, which is probably good.

It is causing some drag though. Yesterday, before putting this into action, I recorded six efforts. Today I recorded 40. I did some eyeballing of the time it takes to shift gears, and it looks to be about 30 seconds. Once I get accustomed to using it regularly I can collect some real data for a better estimate. But if it’s 30 seconds and I recorded 34 extra tasks, that’s 17 extra minutes of my work day spent on tracking tasks. That’s a fair bit. If could get the shift down to 15 seconds, that would only be 8.5 minutes, which sounds more reasonable.

There are other benefits to this. Recording things today I found that stopping to record what I was doing made me more mindful of what I was doing. It helped keep my head in the here and now. I also started tracking my moods today, good and bad. It made things easier to let go of. I would get frustrated with something, stop to record that I was frustrated with something, and that action stopped me from feeding the frustration and made it easier to let go of. Data collection seems to be rather Zen.

Sort of. When the train to take me home rolled into the station I was so caught up in adding more things to my data tracking software that I forgot to track that I had gotten on the train. I decided not to worry about it. Instead I just took it as a sign to stop data coding for the day and just chill. This is not the priority right now.

Venting III

Going back to work today was kind of stressful, although it started to lean toward the screwy. It took me three tries and over forty minutes to get my computer started today. Of course, when I finally got it started it told me I was ten minutes late for a meeting. It was one of those meetings that just sort of drones on. Then I went back on the panic treadmill. Tuesdays I have to run the Batch, which we weren’t sure was going to work because we weren’t sure we had any data (thankfully we did and it did). Then I had to modify another program to handle the fact that we were off Monday. Everyone was freaking that it was only returning two reports, even though it’s really an obsolete system (we’re just running it because we don’t trust the new system). I fixed it and got it to return 749 incidents, which pleased everyone except the guy whose turn it was to review them. Then I had to get the data from the old system combined and ready for upload into the new system. Of course, it was complicated by someone (again) checking out a document and then leaving. Our organization has been using SharePoint for over a year now, people, time to get the hang of it. All of the panic got interrupted by the mandatory ethics training. Normally they entice you to participate by handing out candy, but this year they seemed to be handing out candy to random people after someone participated. I got stiffed, but several others had their candy handed out to other people. It seems to me that there’s an ethics violation in there somewhere. 😉

It wasn’t all bad. I met my new supervisor today. He seems like a nice guy. I was going to say I had forgotten his name already but I remembered it before I finished typing the sentence. I pity him. He has to get up to speed on all our data systems, the different tasks of a dozen people, and do it all in the middle of switching over to a new system that is full of bugs. Maybe I’ll get donuts as a welcoming gift tomorrow. Or rather, maybe I’ll remember to do that.

Today was the first significant test of Log Life. I tracked my commute with it with only a few minor bobbles. Note to self: don’t walk across the street while logging your life into your phone, or you may end up logging your death. I was able to log every step of the commute easily:

  • 8:13 left the house.
  • 8:17 got to the bus stop
  • 8:22 got on bus #05 5943
  • 8:33 got to Shady Grove metro station
  • 8:36 Train car #2027 left the station
  • 8:54 got to Bethesda metro station
  • 9:00 got to the Safeway
  • 9:06 left the Safeway
  • 9:15 arrived at work

And similarly on the way home. As noted above it even let me easily note the numbers of the buses and train cars I ride, something I’ve always been curious about. There are a few details to work out. What do I want to count as arriving at work? Today it was getting in the building, but going forward I think it should be when I turn on my computer. I also need to figure out when exactly the time stamp occurs on a log entry. It can still take a little while to code things in, so there’s a lag between when I start and when I finish. If I want to time something accurately I need to know if the time stamp is made when I start entering or when I finish entering.

Today I finally got around to cleaning out my task tracking data at work for the start of the new fiscal year. The software I use (Task Coach) hasn’t had any problems with the data, but I don’t want to push it by filling it up with three years worth of Batch runs. I took the opportunity to a new category for “wasting time” and set up some “projects” under it to track things like chit chatting with co-workers and cleaning up my desk at the end of the day. That should allow me to track every minute of my work day, and then I can combine that data with the rest of the data I track on my phone.

Fair warning to everyone: I have lots of projects that show a lot of enthusiasm at the start like this one, and then crash and burn. I don’t know if that’s what will happen here, and I’m not going to worry about it. Right now I’m in test mode: try it out, see what the issues and consequences are, that sort of thing. But my priority right now is the habit forming course I’m taking, and if there is any conflict whatsoever, the personal informatics is going by the wayside.

Oh, and one last odd thing from the commute home. I’m in mindfulness mode as I try to be when walking places, and I’m coming down the six story escalator into Bethesda metro station. What do I see but a huge crowd at the bottom of the escalators. I was a bit nervous because that’s the sort of thing you see when they’re shutting down the station and your commute is about to take two hours. Both of the other escalators were fenced off and shut down, but people seemed to be expecting to be able to go up one of them soon. I’m not sure why they would need to shut them both off at the same time, but the metro station escalators are an enigma. My commute was fine, but not so for the orange line. Someone got hit by a train and they shut down a whole station, with bus service for a connector. Prayers out to the poor soul hit by the train, and here’s hoping my grand-supervisor didn’t get stuck in it.

While I’m at it, prayers out to a family member as well. You know who you are.

Bicycle II

A while back I was talking to some people at work about bicycling to work. I said there was no good way to do it for me, but one lady said you could get on one of the Rock Creek Park trails in Rockville. That leads to the Capital Crescent Trail which goes right behind our office building.

Earlier this week I finally got around to looking into it. I found a way to a south bound trail in Rock Creek Park from Shady Grove Road. Getting to Shady Grove Road from my place is a problem, though. You have to go down Oakmont, which has bad or no shoulders, no sidewalks, and lots of cars during rush hour.

But I figured I might as well give it a try. So I got on my bike and rode to Rock Creek Park. Oakmont was not fun. The should looks a lot worse from my bicycle than it does from the bus I ride in the morning. There were cars, driving slowly behind me or zipping by rather close, but only a few. It would be nerve wracking to have that going on the whole way down Oakmont every morning and every night (in the dark, no less). I did have some false starts along the way. I misread Needwood Road twice: once from Google Maps so I was looking for Needlewood Road, and once when I rode past what I thought was Norwood road. Then I rode past the entrance to Rock Creek Park thinking it was part of the Needwood Golf Club.

The killer was the trail itself. Hilly, which I can deal with, but with big chunky gravel that my bike can’t. My bike is a city bike. It’s not a racer, but neither is it a mountain bike. I was nervous going down that hill, and I don’t want to think about trying to get up it late at night without knobby tires. I am certainly not up to commuting on a path like that. I had to go east a ways to get to the trail itself, so I still had at least 15 miles to go.

Still, it wasn’t all bad. I got to do the first test of my Log Life application. The GPS is sucking, giving me 1,275 meter accuracy, but that’s not Log Life’s fault. I also screwed up starting it both times. I didn’t start it going to Rock Creek Park until I was halfway down Shady Grove Road, and I didn’t start it from Rock Creek Park until I was back on Needwood Road. Still, I did log 58 minutes of bicycling with it. Which is another killer. If it takes a half hour to get to the trail, I don’t want to think how long the rest of the commute is going to be.

As for the data, I keep thinking of things I could log. I think I will continue with just logging the movement and ideas for the next week or so. As I think of other things to log I will set them up in the application. Then when I am ready to move forward I can wipe the log and start fresh.

Data I

So I’ve been looking more into the Quantified Self Movement that I mentioned yesterday. Fascinating stuff. To me, at least.

I downloaded the app I linked to yesterday (Log Life). It actually has broader applicability than just life logging. The designer of the app (Richard Ehmer) was trying to solve a typical problem with personal data collection: it takes too long. Indeed, that’s why my earlier efforts in this direction failed. Say I want to log when my bus shows up in the morning. When the bus gets there I have to get out my phone, a pen, and my notebook; then check the time and write it down in the notebook. If I want to log my entire commute I have to do those same actions eight times. With the Log Life app, I just get out my phone, click on the app, and click on two buttons I already set up. That automatically logs what happened, when it happened, and where it happened (by hooking in to the GPS). Plus, it can easily be modified to track all my travel, whether I walk, drive, or ride my bicycle.

So the app has broader use in entering in any data quickly. He has a couple interesting examples on his website. In one case he logged various characters, events, and phrases in a TV version of Treasure Island. Not only was he able to track about 30 different things without pausing the movie, but he developed the button definitions to do so on the fly as he was watching it. He also tracked the data on the Jeopardy episodes featuring Watson. He was able to track the dollar value of each question, each attempted answer an whether it was right. (The data suggest that Watson won not because he was better at trivia, but because he was faster on the buzzer).

My favorite thing about the app is that it will export all the data to csv files. Sure, it can do basic graphs for you, but they’re all pie charts, and good God do I hate pie charts. With the csv files I can analyze the data myself. Not only that, I can combine it with other data sources. I’m already tracking my work activities pretty closely. I could pull in weather data and compare it with mood tracking on Log Life. Considering all the data available on the internet, I could probably find out which of my activities have the biggest effect on the price of tea in China.

This is not something you can do with everything I looked into. There are some very nice physical monitors you can get, like Fitbit and BodyMedia. You wear them constantly (yes, that means 24/7) and they collect data on all sorts of activities, from pedometer style step counts to galvanic skin response. From your motions while in bed they can tell when you go to bed, when you actually get to sleep, and how often you wake up in the middle of the night. But they don’t give you the data. The data collected by the monitor uploads directly to their websites, not bothering to stop off at your computer. I especially like how Fitbit says “the data is yours, we will always offer free accounts.” You can’t get the data with the free accounts, just a bunch of pie charts. You can only get the data with the premium account ($50/year) and even then they only give you summary data. I was very interested in buying one of those, but I’m not going to buy either one if I can’t have my own damn data. And what are they doing with that data? It seems to me that their business model must rest on selling that data to someone (a la facebook and Google) , but who?

A cool gadget I did find was the Vicon Revue Sensecam. You wear it around your neck and it takes pictures whenever it senses a change in motion or environment. You can use it to make a time elapse film of your whole day. It does cost about $500, but I happen to have that much handy. While it is cool, I’m not sure I want to buy it. I’m a data guy, how am I going to get data out of all those pictures? Maybe I could train a neural net to recognize certain features of the pictures…

Some of this I found through the link I posted yesterday to quantifiedself.com. But I also got some of it from a paper written on analyzing different systems for collecting this sort of data, which they call personal informatics. It’s by Li, Dey, and Forlizzi from the Human Computer Interaction Institute at Carnegie Mellon. It’s got all sorts of fascinating stuff in it, like Benjamin Franklin’s 13 virtues (which he tracked his adherence to for 60 years), Nicholas Feltron’s yearly reports on himself, and FlowingData (I’m not sure yet, but it looks to blow Information is Beautiful out of the water, not that it’s hard to do that). I haven’t finished reading it yet, but maybe I’ll have time tomorrow.

An amazing vista has opened before me, but I can’t dive into it yet. I’m doing an online habit forming course, which emphasizes one habit change at a time. I can’t dive full force into gathering mounds of data about myself and give full attention to the one habit I’m supposed to be working on. And the lesson of the habit forming course is that I shouldn’t. I’m not sure I can resist cheating, though. I may try just starting with tracking my travel and ideas (Log Life looks to be good for making short, categorized notes), but be ready to ditch it if it starts to interfere with my new habit.

Random IV

Uh oh. I just encountered the concept of the Quantified Self Movement. I have thought about trying to do this sort of thing before, but now I find out there is an app for that, not to mention a gadget. I feel myself getting sucked it, and I don’t want to resist.