LEWT

London Exploratory Workshop in Testing

View project on GitHub

The London Exploratory Workshop in Testing.

LEWT is a peer workshop. It's a safe-ish place for a small group to explore ideas in software testing. LEWT01 was in 2005, and LEWT12 was in 2014, but it is currently inactive. This site exists to explain some of the ideas behind what we did. None of this is gospel: Other peer workshops do other things, and yours should, too.

What it was for

I started LEWT because I found that peer workshops gave me more insights more quickly than any other format. My experience was with LAWST-format workshops, and though the LAWST handbook was a crucial source*, I wanted things that LAWST didn't. LAWST aimed to "crystalize", while I wanted to explore. I particularly valued diverse opinions held flexibly by interested people – and I wanted to avoid agenda-setting polemics. I sketched out something that gave made the group responsible for the content, the composition, and the shape of the day. We had an audio recording rather than a scribe. This had the happy side-effect (also an aim) of making workshops easy to organise. We didn't need a content owner, we didn't spend time deciding who might meet our standards, we took moments to decide who was on first.

I was influenced by Open Space formats, and (oddly enough) by the TimeOut Guide to Country Walks. I expect that hindsight gives more reason and shape than was actually present.

Who comes, and what we talk about

I'm often asked – by organisers – how I select people for LEWT.

Short answer: We don't.

Group composition is important. I want a diversity of ideas, but if I selected the group, I expect that the group would reflect my biases. So /I/ don't select anyone. /We/ have a very simple approach to application and invitation - if someone asks if they can come, then they can. Done.

We usually – but not inevitably – set a topic beforehand. It's usually one word. I may suggest some, I ask prior attendees to suggest others, and I pick one.

I ask everyone to come prepared to offer a talk. People generally – but not inevitably – indicate their angle on the topic to the group as they claim their space. Talks are insubstantial; 10 minutes long at most. The important stuff lives in the discussions after the talk.

We vote on which talks go first. We timebox the talks so that we have a chance of getting through most or all of them, but later talks may get less time. Sometimes people add or withdraw talks during the day.

We don't have a content owner deciding what gets attention or priority, we don't have a scribe making public notes, we don't have a mission. We do have a facilitator, and agree at the outset to be facilitated.

I don't select people, and I don't select talks. To the extent that it matters, LEWT selects them. LEWT is a group, but it's not my group. It is, I hope, its own.

Strategy

Initially, I wrote this page. It sums up my approach, but it neglects one important underlying consideration that was so obvious to me that I didn’t think to include it when I wrote the page. I can see, now, that it was neither obvious, nor well-shared – but it still gives shape to most of how I run (ran?) LEWT.

The thought is this: at a peer workshop, I should consider everyone my peer. For the duration of the workshop, I will attempt to listen to – and question – anyone who I share the room with, regardless of whether they have more or less experience, or whether I generally consider their work good or poor, whether I am fascinated, bored, repelled, awestruck or confused. Taking that position might open my mind to more ideas from more sources – and if that’s the general attitude of most participants, then a peer workshop is a broadly safe place for me to reveal my honest experience and to play with novelties. When I first tried peer conferences, I hoped that everyone walking into such a workshop would give each other, temporarily, the healthy and questioning respect they’d give a peer.

I had assumed that sense was a central guidance to peer conferences – even if, in practice, it was occasionally hard to see such respect in action. However, I’m no longer certain of this; when I’ve shared my position with other peer conference organisers, it has been (generally) either alien or less important. I think this gets hard with > 8 people, and is pretty impossible with >15. A 25-person room will naturally form groups, gurus, acolytes and pariahs – so it’s ludicrous of me to expect larger peer conferences to work this way.

Note: The facilitator is /not/ a peer. The participants give the facilitator their attention, and their permission to stop and start them, in pursuit of a greater goal then their own individual airtime. The facilitator accepts their temporary status, and returns the favour by serving the group and putting his or her own needs aside.

Special cases

If someone's interested enough to ask to come to LEWT and to give up their time to be part of it, then they're in - whether they 'fit' the group, or not. We have had people who didn't fit, and sometimes they've been wonderful contributors, sometimes they've triggered good conversations and interesting realignments. No one has walked out yet. A few participants have complained about others, and I can deal with that as facilitator if something is said early enough. I sometimes find my own comfort challenged – but I don't think it's my role to exclude someone, and I'm sure that the group is muscular enough to chew someone up and spit them out if it absolutely has to.

But there are always wrinkles.

  • some people think they have a right to a seat: Currently, I ask prior participants to set the theme and the date, so they know before anyone else. This gives them precedence, but if they don't take the opportunity, they don't get to go.
  • too many people for the group: I tell people that there's a cutoff, what the cutoff is, and that people who apply when numbers are under the cutoff can come, and people who are later can't come. Personally, I think the max size for any peer group is rather under 20.
  • too many people for the room: I find a new room. Hasn't happened - our usual room takes 18 at a pinch, and that's us.
  • wrong people: who am I to judge? However, if someone applies out of the blue, I'll talk with them so that /they/ can judge if they're the right person. Usually their judgement is sound.

So, that’s the strategy and setup – and I wanted to get it out of the way before the tactical and practical bit. Most of these tactics are, necessarily, from the facilitator’s point of view, because with LEWT I’m organiser and (for most of the time) facilitator.

Breaks

As long as possible without losing momentum and direction. Proper, multithreaded conversation happens in the breaks. The “talks” are a primer for the discussions, the discussions a primer for conversations – and connections and ideas grow from those conversations.

Getting everyone’s attention focussed from chat to the group

There are clutch of approaches. Most work, most use sound or visual cues. I pick up whatever (physical) sound effect I’ve not used recently. Singing bowls, thundersticks, jingle toys. It gets to a point where, when everyone’s concentrating, one has only to pick the thing up to make people switch focus. My favourite was the vuvuzela – a disgustingly loud football horn. I don’t remember blowing it at all (except to try it out).

Timing

Allow the clock to rule, allow the room to override the clock. Don’t worry about going short. The room will need to regularly be reminded of the time available as the stack builds up and time burns down.

Ordering

The room gets to decide what goes early (the facilitator gets a deciding vote) – so topics at the end usually get less time. This can make them more focussed, and the speaker will often be able to tune what they have so that it suits the attention of the room.

Set-up

Everyone should be able to see everyone else’s face, all the time. Other than that, don’t be precious about room layout, drinks, stationery, power supplies, matching tables or any other fripperies. Indeed, the more informal, the better. Help participants to feel comfortable, not coddled, and certainly not privileged.

Visuals

I strongly discourage slides, and encourage flip charts. They’re more immediate, more interactive, and less goes wrong. I prefer flipcharts to whiteboards, as they’re more permanent and one can flip back.

Recordings

I record LEWT (I use a Rode NT-4 stero mic, if you're wondering). I record because I think there should be a recording, one we can revisit and remind ourselves what was said, who said it, what was said before, what the reactions were. To listen at leisure, or after a long while. I record, also, because I can; it's something I do occasionally for money. Everyone in the workshop has access to the recordings, which I split and tag as MP3s (and what a pain that can be – but Fission and various hefty DAWs help). However, being recorded changes something about how one communicates. More so if one knows that posterity is public, so I ask the group to not distribute the recordings beyond those whe were part of that particular LEWT. Sometimes, we've all agreed, and have allowed a recording to be made more public. But LEWT isn't a performance, and I don't want people censoring their interactions becuase they're worried the reaction of an imaginary someone outside the context of the room.

We don't have a scribe (someone who writes up what's going on, as it's going on, in a public way – a flipchart, big wall or blog). A scribe is yet another non-peer role – and their record is necessarily both partial and biased. We don't have a transcript. They can be pricy, and don't capture nuance or (easily) the speaker. Also, my recordings are sometimes rather indistinct.

On facilitation

I've been the primary facilitator and instigator of the LEWTs to date, so while this is LEWT's approach (so far), it's written fro my own point of view. Yours, and your group's, will be different.

When I'm facilitating, I try to do the job with as light a touch as possible - basically I keep a queue, keep my eye on time, and try to help the group stay within the discipline of conversing in a way that lets everyone talk, and everyone listen. Even that, however, requires my complete attention on the room - which means I don't make many notes for myself or contribute much to the conversation. As a facilitator, the people who give me problems are those who assume their contribution is more important than the person who currently has the room's attention, the people with one thing to say and a big personal stake in having it heard, and people who stop listening after someone uses a word that is hot (or dull) for them. I'm sometimes a problem if I get involved, and I'm lucky that people help me rein myself in if I get out of hand. But problems are few and often easy to deal with if one has a feel for the tolerance and firmness that suits the mood of the room (the whole group, not just the loud participants). The relatively-fast turnover of topics helps, a lot. Also, I find that the more diverse the group, the more it offers guarded respect to each individual: our two-people-with-less-than-two-years-experience thing helps with the diversity.

Ideas…

  • Ask the room to accept you as someone who will regulate the ebb and flow. Don’t direct (or dictate) the content.
  • Accept that, as facilitator, you’re not really at the workshop, and give the primary part of your attention to emotions of the people in the room, not to what is being said.
  • I find that expression and body position will tell you whether someone has a new point or a follow-on (and if not, just ask), so I think that k-cards in something with <20 people are a constraining gadget.
  • Discourage bad behaviour more than the person who is behaving badly: Firmly and clearly block people who are being bullies, then swiftly forgive them and allow them a chance to redeem themselves in the eyes of their peers.
  • I bite my tongue (metaphorically, mostly) to stop (my) witty interjections; they’re not usually that great, and it’s an abuse of the role the room has allowed me to take. For the same reason, I don’t usually ask many questions – but I don’t absolutely exclude myself, either. If, as time runs out on a topic, you give participants the chance to pull their questions or comments to let other questions be asked, they might just do it.
  • Don’t fear dropping a person from the queue – it’s your job. But don’t drop them slyly, either.
  • If everyone speaks at once, I need to decide when and how and whether to stop them – and if people only speak when their feel they have permission to speak, I’ve done it all wrong and need to shake up the room. Stay between these extremes, let people (including yourself) be human, aim for fine chat, and you’ll have done a job that anyone should be satisfied with.

Practicalities…

  • For each new topic, I try to remember to announce the topic and speaker, ask how much time they want to talk, support them no more than they want, and to ask the room to thank them at the end.
  • As someone starts their topic, I split the audio recording and also write down the start time (to the nearest 15s…), the time the speaker’s asked me to give them, and the time we’ve all agreed to spend on the topic. I write those as absolutes, not relatives, because calculation takes your attention – (ie 10:03:15, 10:13, 10:33). My laptop clock is always in view.
  • I record audio – and this also keeps track of elapsed relative time (i.e. 0:17:30 since the topic started).
  • I keep track of the timing info and the current queue on the same topic card that I’ve pulled off the wall – the card that started with a topic title and ended up covered in sticky dots. Keeping track of the question stack/queue is easy – it’s a list, sometimes with indents and squiggles. If sub-topics are spawning more sub-topics, do ask the room if they want to go deep or wide. *Every few of questions, I’ll tell the room who the next 2-4 people on the stack. If we’re in open discussion, and I feel the room needs to move on, I’ll catch the eye of whoever is speaking, breathe in as they finish a point, and indicate the next question by pointing to someone and saying their name.
  • Use audio and visual cues together – people thinking need as much help as possible.
  • Name cards can help your own flow.
  • I don’t tend to give much leeway to and extended back-and-forth between speaker and a single interlocutor.
  • If you can, find someone else to do the distracting mid-workshop logistics (i.e. who’s eating what, taking calls from late people).

Success or failure (pick your own definition) is mostly down to the group, not the facilitator – but you are, as Jerry Weinberg might say, responsible for your reactions to the group.

Progenitors and successors

Ideas and groups evolve in a Lamarkian, not Darwinian, fashion. LEWT owes its form and existence to all these groups, and I'm grateful to those groups, their facilitators and participants.

Groups we had in mind when running early LEWTs include: LAWST, STMR, WHET, ExTRS, WOPR

Groups I've facilitated on LEWT lines: SWOPR, Let's Test EWT

Groups who have asked me for advice: DEWT, MEWT, CEWT, SWEWT

Others: TiTAN, Swedish WST, more...

Source

This document is an aggregation of emails I've written to organisers (and putative organisers) if a bunch of groups. I wrote all of it, though you'll find some of it shared elsewhere in earlier articles. I'm making this open source / under creative commons, so use it as you wish. I'll blog it in bits, but the live version is here – and I'm accepting changes.