Grey bar Blue bar
Share this:

Sat, 29 Sep 2007

The myth of the expert

Something we preach very strongly in our training is the importance of an understanding of the underlying technology / application / issues, and being able to dig into the core of an issue, not just try a trick or two and move on. Sadly, most people don't see it this way.

It's also somewhere between sad and frustrating for me that there seems to be an over-abundance of so-called "experts" in our field. While this isn't an issue for those who have a deep understanding, the fact of the matter is that for many of our customers, their key competence is their respective industry, and not information security.

Of course, this leads to much snake-oil and other uglyness...and to increased frustration for those of us who actually *are* trying to help our customers and add value. Let it be said right now that I don't by any measure regard myself as an expert on all things information security, but I'm more than happy to tell people when something is outside of my field of expertise.

I found an interesting piece in a book I'm currently reading called "Way of the Turtle" by Curtis M Faith - this is in the context of traders and the markets, but is more than applicable to our industry, practically verbatim. The snippet, from a sidebar in the book titles "The Myth of the Expert" follows.

-snip-

The "don't optimize" counsel is an effect of what my friends and I like to call the myth of the expert. Unfortunately, in most fields the number of people who really understand what's going on is very limited. For every true expert, there are scores of *pseudo-experts* who are able to perform in the field, have assembled loads of loads of knowledge, and in the eyes of those who are not experts are indistinguishable from the true experts. These pseudo-experts can function but do not really *understand* the area in which they claim expertise.

True experts do not have rigid rules; they *understand* what's going on, and so they do not need rigid rules.

Pseudo-experts, however, *don't understand*, and so they tend to look at what the experts are doing and copy it. They know *what to do* but not *why it should be done*. Therefore, they listen to the true experts and create rigid rules where none were intended.

One sure sign of a pseudo-expert is writing that is unclear and difficult to follow. Unclear writing comes from unclear thinking. A true expert will be able to explain complicated ideas in ways that are clear and easy to understand.

Another common characteristic of pseudo-experts is that they know how to apply complex processes and techniques and have been well trained but do not understand the limits of those techniques.

In trading, a good example would be someone who can perform complex statistical analyses of trades, runs a simulation that generates 1 000 trades, and then assumes that she can draw conclusions from those trades without regard for the fact that they might have been drawn from only two weeks of short-term data. These people can do the math but do not understand that the math does not matter if next week is radically different from the last two weeks.

Don't confuse experience with expertise or knowledge with wisdom.

-snip-

This rocks...I couldn't have said it better myself :>

** CRM114 Whitelisted by: From nick@sensepost.com **

Sun, 2 Sep 2007

Medical Doctors.. bah! hambug..

I've ranted a few times about things i hate about the way we "do medicine". (Doctors are not alone here.. i cant believe that in the age where we operate on the eye with lasers and see production ready nano-tech. we consider yanking teeth with a pair of pliers a reasonable option)

Recently i heard an interview with the head of MS Research where he spoke about some of the same things.. i.e. that 9/10 people are visiting the doctor for the same thing (that new strain of flu going around) and that we could help alot of things with a simple "if you have a fever, and a runny nose and red spots today, u have the latest X going around.. take 2 of X and get some rest". This would handle the majority of the ppl walking in..

(actually, as yet another diversion to a fairly arb. post - i get bothered whenever i see gross inefficiencies like this.. arriving at new york airport for example u get to the end of this passage, and theres a lady standing there screaming.. everyone left please.. everyone left.. the fact that this lady is doing this at all is silly.. if she ever did it more than once in her life without realising she could replace herself with a sign and an arrow is ludicrous.)

Other things about the way we doctor bother me.. like that lack of liability.. about 5 years ago my dad walked into a doctors rooms with the flu.. they discovered that his blood pressure was insanely high and immediately medicated it down.. of course, they never checked his medical history so they had no idea if he was running with insanely high blood pressure his whole life.. dropping the pressure was a shock to the system which caused him to have a stroke.. it bothers me first that this happened.. it bothers me even more that in subsequent discussions  no one ever said "oops.. we blew that one"

I have heard horror stories of mis-diagnosis and perfectly incorrect treatments being doled out.. but its wrapped in a culture of almost non performance.. i.e. if i take mar car in to a mechanic because it doesnt run, and they cant make it run, then i dont pay them.. u can go to a doctor for X, comeback with Y and Z and the thought never comes up that i never got what i paid for..

Ill end this pointless rant with a scary ponderation i had a little while back.. If you take a medical student who scored 50% on every test he ever took all through his medical education, he eventually graduates and starts practicing.. so.. when your kid has this whooping cough, and u go to the doctor, you are actually going to a guy who has historically shown that he will misdiagnose 1 of every 2 cases that walk through his door..

this is scary.. and so just in case ill make sure im only ever the 2nd guy to walk in to a doctors rooms.. keeping on even numbers should help..

Thu, 23 Aug 2007

MTBF and Light Bulbs..

Some of you will know that i finally moved out of the shoe box i lived in for 6 years and moved into a house (about 3 months ago) Since then i have replaced 3 different light bulbs at different places in the house.. Now this made me start thinking.. Surely when the house was new, they fitted in all the bulbs as brand new.. Now some sections of the house light a series of 4 or 6 bulbs at once.. yet there appears to be no link at all between "sibling" bulbs and their life-span..

It (only) then occurred to me, that ive never heard someone say "well.. its coming around to light bulb month.. going to have to change all my bulbs soon?" because the bulbs show no consistency at all. It is probably a good thing, since you dont come home and find the whole house in darkness, because all bulbs went off simultaneously but if this is design, then the bulb should have some sort of label indicating where on the curve it sits.. and it doesnt, which is pretty curious..

It sounds really strange that any piece of consumer engineering can have such an in-determinate mtbf.. there must be a better explanation and since we have such smart people who work here (and or read this blog) im hoping someone will save me the googling.....

Mon, 20 Aug 2007

2 Un-related thoughts.. on Echelon and the recent Skype Outage..

I suspect somewhere there exist cardinal rules of blogging which would state that using a single post to make 2 completely un-related posts is a no-no.. I will now promptly ignore it 2 push out 2 random thoughts that came up..

Echelon and Echelon spam..

While watching the Bourne Ultimatum the other night the usual "echelon"esque scene played out.. Guy on phone says keyword.. pan to NSA/CIA type building.. computer drone type person screams something like "we have a hot one"..

Now i admit to knowing very little about echelon and how it actually operates, but figure if i lived in the states (where i believe local calls are free) i would have my phone generate echelon spam when not in active use.. Concerns about tying up your line? use it as hold music.. Effectively a bunch of people worried about their privacy should be able to inject enough noise in the system to render it less useful.. it sounds ferpeclty feasible...

Skype and the recent Skype Outage..

skype_logo.png
"

So lots of people wrote about it (before and after skype's official response).. Basically on August 16 Skype had a major outage.. this is old news.. but what is really interesting (partly because i only recently finished Talebs "fooled by randomness" is the law of un-intended consequences coming into play.. Skype by many accounts is well engineered and the skype network is built to withstand spikes in usage.. Even its peer-to-peer net has what they call self healing capability.. So what took skype down? a massive botnet? a co-ordinated attack? Windows Patch release cycle.. Turns out that skype was not able to handle the number of machines that all simultaneously re-booted with the last windows update update.. This apparently caused a chain reaction and the rest is history.. its really interesting because with any reasonably complex system, there are always matters beyond the horizon, that are near impossible to see coming..

/mh

Tue, 14 Aug 2007

mh.blackhatFeedback(Side-jacking, Hamster)

Ok.. so its a lot later than i promised, but i did mention that i would post some feedback on some of the talks i ended up catching at this years BlackHat. By far the talk that grabbed the most press was the Erratasec talk on Side-Jacking.

Essentially the researchers demonstrated a tool (hamster) that allows an attacker on a shared network (wifi was used as an example, but i guess any shared medium would suffice) to hi-jack users accounts by sniffing their session-ids.

The confusing thing about the talk (other than the fact that in discussions about it, people seem to completely confuse the concepts of a cookie, a session, a cookie-expiry, a session-timeout and other basic HTTP concepts) was the crowd reaction to it. People were amazed and clapped loudly as Robert Graham "impersonated" some other user whos session-id he captured.

It felt a little surreal to see the number of people who were visibly shaken at the thought that a person who captured their session-id could impersonate them. This is something we have been teaching in our Hacking by Numbers course since 2000 and has had a solution (use SSL) for longer than time (internet time at any rate).

It was totally confusing to see it being called a "Web 2.0" problem" (and worse to see "Hamster plus Hotspot equals Web 2.0 meltdown!", and at a point i even heard "Well Google are ahead with Web 2.0 since they pioneered it with GMail, so they have a fix but other Web 2.0 sites are all broken". Stealing someone's session-id on a shared network when SSL is not being used is hardly a Web 2.0 problem...

Of course this didnt stop the press / blogosphere from exploding the story and i believe news of it made non-trade-press too with appearances on BBC & CNN? It truly blew me away, disillusionment++

(To be clear.. this casts no aspersions on the researchers, but the response to essentially an ancient problem..)

At a point, Robert did his demo live on stage.. which made me wonder if his Ferret/Hamster which was going to allow an attacker to exploit Web2.0 sites actually protected itself from other "Web2.0 attacks".. (it doesn't.. so essentially if an attacker is running hamster on your wifi connection, you should be able to attack him with about 5 lines of pasted text in a telnet session)(ill post details in a follow up post..)

/mh