blog refresh

I can’t believe it’s been a year since I posted anything. 2011 was a depressing year, to be sure, but some good things did happen:

  • Got my music into a hit video game, Starwish
  • Resigned my job to get the Ph.D. completed. Estimate: 18-24 months to go from today.
  • Completed a 1000km rally on the rebuilt ’78 CB750, but just barely – she needs to go back into the shop this winter
  • Regained confidence in programming through some short-term hackfests
  • Started prep for the related Introduction to CouchDB Development course, kicking off next week
  • Vacations in both Buenos Aires and Singapore, including time with good friends in each
  • Helping lead and teach in some online gaming communities

I’m entering a very changeable 2012, personally, and I’m not sure exactly how it’ll turn out. At least I have a big list of things to get done!

presentation at icel 09

so a couple of weeks ago i presented my third academic publication, this one titled “persisting chat for communities of practice.”

joan at icel 2009

joan at icel 2009

in layman’s terms, it’s a new logging system for online group chat (irc, jabber, etc.) with special integrations into non-synchronous systems such as web forums (academically often called asynchronous learning networks), blogs, etc. the goal is to make chat less transient and throwaway, to promote it to a first class citizen within the wide variety of mechanisms that can be used to help communities. it’s especially designed for communities of practise as described by lave and wenger out at xerox parc way back in the day. go read the linked article, it’s not half bad for wikipedia.

the system will be released under an open source license later this year. if you’d like to get involved, comment here using your real email address and i’ll get in touch.

CouchDB benchmarking followup

I wanted to post a public thanks to the great minds over in freenode #couchdb, including Paul Davis and Jan Lehnardt for helping me learn more about CouchDB, and helping me investigate the performance issue I posted about last time. In fact, they both posted on planet couchdb with some thoughts about benchmarking in general.

I wanted to share my reason for benchmarking CouchDB in the manner I did. The fact is that it was entirely personal in nature, as in trying to make my own dataset and code as fast as possible. I’m not trying to generate pretty graphs for my management, for publication or for personal gain. This work comes out of having to load and reload fairly large datasets from scratch on a regular basis as a part of my research methodology. I need a large dataset to get meaningful results out of the rest of my system, and I was not content to wait for an hour or two each time my system bulk loaded the content.

So the suggestions provided – not using random UUIDs to help CouchDB balance the b+-tree, correctly using bulk insert, and redoing my code to use the official couchdb interface (instead of a hacked-up version using raw PUTs and POSTs) helped a lot.

It turned out that the spike that I was seeing (see last post) disappeared when I randomized the order of incrementing that variable, so much so that 3 randomized runs show almost no peaks. However, when I “scroll” through that variable (increasing the batch size) I still see the peak around a batch size of 3k.

Trying the software on another platform (AMD x64-3500 Debian lenny with a 3ware hardware RAID array, as opposed to a 4-core Mac Pro with only a single local disk) revealed that the peak shifted to a different value, a much higher one.

Lesson? Always benchmark your own application against your own data, and tweak until you’re satisfied. Or, in the words immortalized at Soundtracks Recording Studio, New Trier High School, “Jiggle the thingy until it clicks.”

I suspect suspected that Jan’s article ranting about benchmarking was at least in part stimulated by my experiences as shared over IRC. (I was wrong – see the comments.) They must have seemed somewhat inscrutable — why would someone care so much about something most database-backed applications will do rarely, as compared to reads, queries and updates? My application right now has a very particular set of criteria for performance (repeatable high performance bulk load) that is not likely anyone’s standard use case but my own. Nor is it going to be a worthwhile effort on the developers’ part to spend a bundle of effort optimizing this particular scenario.

That said, Jan also is calling for people to start compiling profiling suites that “…simulate different usage patterns of their system.” With this research, I don’t have the weight of a corporation who is willing to agree on “…a set of benchmarks that objectively measure performance for easy comparison,” but I can at least contribute my use case for use by others. Paul Davis’ benchmark script looks quite a bit like what I’m doing, except the number of documents is larger by a factor of 100 (~2mil here) and the per-document size is smaller by a factor of 25 (100-200 bytes here). Knowing the time it takes to insert and to run a basic map/reduce function on fairly similar data is a great place to start thinking about performance considerations in my application.

Oh, and knowing the new code on the coming branches will get me a performance increase of at least 2x with no effort on my part is the icing on the cake.

Thdavisp. Thjan.

camera dead

You might have noticed that my last few posts have been image-less. This isn’t because I’m protesting visual communication. I love it – in fact, I’m downright jealous of what most other bloggers pull off with the visual design of their sites!

My camera died. Yeah, the “new old” Olympus C-5050 I bought broke. I only have myself to blame. The camera was on my kitchen counter. I reached for it, but scooted it off the counter, dashing it on the floor. The damage could have been worse – only the mode dial came off. I can’t just reattach it using the screw hack someone posted, because the impact on the floor cut through traces on the flexible PCB. And they’re too fine to bridge easily. Besides, disassembling to the point where I could even consider repairing that damage ended up cracking a couple of very delicate plastic tabs, ones designed to hold the whole thing together. I could try and glue all of it together, but…

Someone on eBay has the entire mode dial assembly for $39, plus $8 for shipping to Canada. Soon I’ll have a working camera again, and by then the studio should be warmer, so lots more pictures and music for everyone.

Oh, and I’ll have my first actual honest to goodness conference submission done by then too (deadline: Dec. 18th). It’s been a long time coming but it’s good to have actual research data and prepare it for publication again. YES!

mr. looper^W cooper^W hooper

Portrait of Mr. Hooper, early Sesame Street character.Today, for the first time, I saw this Sesame Street clip in which the human cast explains to Big Bird that Mr. Hooper has died, and won’t be coming back. If you grew up on early Sesame Street, like I did, you may want to brace yourself for the footage’s emotional impact. A transcript is also available if you can’t view the video, or can’t bring yourself to watch it.

My long-time readers know that I’m a major proponent of the pedagogy behind and execution of the early works of the Children’s Television Sesame Workshop, and that one of my side projects helped push them into releasing some of the old content onto DVD. So why is this the first time that I’ve seen this clip? The truth is, when this first aired, I wasn’t allowed to watch it.

Continue reading

joanbits 2008.03

As the snow finally melts away, I start to get busier and busier. Here’s what I’ve been up to the past week:

  • Work: supporting new projects as usual, writing position papers, working on newsletters, technical enablement, beta testing.
  • House: cooking every night (pasta from scratch, ramen from scratch, gourmet hamburgers…will try and post some pics soon), planning garden, staring at wall that needs repairing and trying to motivate to fix it, regular cleaning, indoor gardening…
  • School: Developing axiology, epistemology, methodology for design research approach. Gave guest lecture on internal Wikipedia politics.
  • Other: Dealing with horrendous migraine. Developing novel database application. Attended One Of A Kind show with friends and got fabulous clothing, jewelry, housewares. Reverse engineering synth. Looking at motorcycle today. Petting cat to deal with stress from everything else i listed.

wohbits 2007.12

some neat tidbits from the past 24h:

owning your online experience

ok, it’s probably painfully obvious at this point that I’m a bit of a net free-speecher – or at least, anti-corporate-net-ownership. [ed: I used to have the word ‘libertarian’ here, but this confused people due to its political overtones. I’m discussing an issue, people, not a political ideology. I’m no more Libertarian than I am Republican or Democrat.]

I hate online profiling, detest advertisements (to the point of not owning a TV!) and generally resist having my activity monitored.

Here’s a quick video if you’re not familiar with this concept, told in allegory:

Do something about it. Visit http://freespeech.org/ourweb/ to find out more. I don’t believe this is about republican vs. democrat, or liberal vs. democrat…again, it’s about owning your own online life and content.

wohbits, 2007.10

  • I managed to get my confidence in gaming again, though I am still having hand/eye coordination issues. Katamari Damacy was what did it, and without selling my soul to the PS2. Half an hour of play is enough to last me all day. Talk about a far better game than Kingdom Hearts!  (Really, I just like crossover stories. The gameplay in KH is awful. Talk about a button masher.)
  • Oh, yeah, I’m talking at CASCON 2007 this year. I’m presenting tomorrow on Education in the 21st century, and how new technology affects how we teach, think and learn. My focus continues to be on affordances of synchronous environments. IRC will be a big part of the presentation, which focuses on our GRAIL project and research @ U of T’s OISE. Research conducted under the guidance of Dr. Clare Brett.
  • Few weeks ago Chris and I started work on the workshop & garage. (OK, he’s been doing most of the work. But I am helping!) I snapped some arsty shots of the foundation in progress, and am feeding them into Lightroom now. I’ll have the pics online shortly, once I’m satisfied that any of them are any good.
  • I’ve realized that the only reason I haven’t been entertaining others more often at my place is a lack of reaching out to people I know (and would like to meet) in the city. Intentionally sticking to “othered” space tends to result in this sort of isolation. Perhaps it’s time to kick off Friday night “red wine and games” again?

lebanon from a lebanese child’s^Wadult’s view

Yesterday, someone who randomly shows up on IRC in a channel I’m in – a 17 year old from Lebanon, living alone, his father in another land – posted a story I felt deserved a wider audience. I’ve love to hear your thoughts – as would he. I’ll make sure they get passed along if you post them here.

He apologizes in advance for his bad English. (I think he’s doing just fine – far better than my Arabic!)

 The Story of Lebanon

come here son and sit down and let me tell you about the problems in the country. one time my dad told me the same thing.

Long, long time ago, god made the planets, he made this tinny country and he named it Lebanon. its environment was gust like the magic. Its men was its real men. How clean was the sea, washes its mountain’s feats. What a good ______ that came from its magic, the grapes where green and fresh, the apple was  THIS size.  The fisher throws the net, it gets out full. It wont finish from holly days, dancing, dabkays, and playing. The amount of the birds singing, leaves with a tired voices at the end of the day. People sleeps at night keeping the doors opened. There used to be friend ships, safe, with love and sharing. The humans used to be peaceful going to his work with smiles.

My grandpa died and did not continue the story. years passed on this story, and my father continued it to me.

He told me suddenly like the speed of the light it came as a touristic country. Its name became east Swiss . Its night lighten like the day. The people are coming and going, planning where to spend there night. The trourist used to come and stay for the whole summer, the beer and food. The candies and the junk food. The living was great and safe. No religions nor clan parties. We used to agree with each other. That’s our houses. No singer in this world did not sing for Beirut. “Beirut you peaceful living” ,”Beirut you good of the world”,  “Beirut you opened minded from street of sool sool till manara”. “ beirut your beautiful on your friendly on rawsheh and the olives”. From the mass of songs, its looks like they have hit it an eye.

It doesn’t know from right or left. It turned to worse and worse suddenly. Its men became hatefull, when they sit to gather smoking and talking about people, when they have the sign, they start killing each other. Each became having 2 homes. first is on somewhere, and the dungeon is his second home. People became homeless no  first nor second. People objected and said no, we could not know who’s fault. Why did this country got ripped like this??!?!?!?!?!!! The judgers’ sake puts on other’s wrong. And we are getting the results with the civil war. Suddenly came in the strangers between us and Israel attacked. That’s my cousin or that’s my friend. but anyhow the people destroyed the country on them selves . Every thing became expensive. 100$ for the gas you take little from it. To get your breads. You have to wait an hour on the line, men and women in this line on the cold and the hot, someone tries to get cross the line with a gun on his wrest.

On the days of your grandpa, our doors used to be opened. On my days its still the same, but this time its broken. I don’t forget that story, I don’t forget that moment, he turned off the cigar and told me am going to sleep. After long time, I called my son at night, so at least I can say the things he told me. I told him the cannon has stopped, we did not know where and why. When they stopped the cannon they bombed us the second day. The problems came with us. There where no enough no mercy. Over it we pay taxes. About the schools, its same as the gallon of oil.  In order for your son to get education you should first sell the house. The citizens went crazy. The dreams became like a visa.

I wish for you my son, to continue the story to your son. Don’t continue it in Canada, continue it in here, in Lebanon. God bless you, and good night.

Omar Al_ Natour