Well, it's not exactly Amazon's recommendations (which have always been abysmally wide of the mark in my case) that someone has chosen to make into giant interactive pictures; it's information contained in the 'Customers who bought this item also bought' links. Why not take a look here at the interconnected cloud of books. I can see a tool like this being very useful for figuring out what to read next. (via GalleyCat)

Flushed with success from the overwhelming response to my previous geeky post, I am thrilled to bring you something else that you'll never use, but if you did would save you masses of time.

Do you ever have an XML file which isn't validating because of invisible non-ASCII characters? Yeah? You do? I bet you do! Allow me to help!

cd path/to/buggy/file

Run the following command:

grep -n -P "[\x80-\xFF]" file.xml

That syntax, broken down:

grep
-n - gives the line number of the problem in the output
-P - tells the computer that there is a Perl regular expression coming up. Perl is a programming language. That's about all I know about it. Doesn't stop me from using it.
"[\x80-\xFF]" - the pattern you want the computer to look for. This will find all characters which are in the range 0x80 to 0xFF.*
file.xml - the name of the file you want to search.

Terminal will run through your file, and report on any characters that match that grep so you can gleefully zap them.

Primary source. You should spend time on Stackoverflow. It's nerdalicious.

----------------

* Eh? you say. Characters don't look like that. They look like this: "a", or "d", or, rather exotically, "z". Yes they do, but computers are funny. Computers like lists, and order, and to know exactly what's what. A computer likes to use a system known as ASCII (American Standard Code for Information Interchange) just to be on the safe side, so that if it's handed a character, it can be sure it knows which character you mean. If you use a character on the ASCII list, you know that your computer - and pretty much any computer software - is going to be able to handle it. The ASCII list assigns values to the most-used characters, using a variety of codes. The letter 'a', for instance, is "61" in hexadecimal code.

0x80 (one of the codes we use in the pattern, here) is a computer-y way of saying the number you probably know as 128, or CXXVIII if you're still into Roman numerals, or 80 if you've been looking at the hexadecimal color picker in Photoshop for too long. 0xFF is 255, or CCLV, or FF. Those pesky non-ASCII characters which don't display on your screen and which bugger up your XML are all in that range.

I've always assumed that academic journals are a force for good because they're the keepers of science's most important documents: the basic research papers themselves. Moreover, journals are part of the mechanism of peer-review which keeps science headed in the right (=truth-seeking) direction. I had noticed that many journals were kind of expensive - scarily so for multi-user licences - but without knowing the business model, I just assumed there was a good reason for that. But increasingly I'm learning disquieting things which suggest that some journals have effectively become toll-gates through which vital information is forced to pass. I've also been struck by the argument that virtually all of the research that appears in scientific journals is wholly or partly funded with public money, and yet if you want to see what your taxes paid for, you have to enrich a private company first. And worse still, scientists themselves have to pay to see each others' publicly funded work.

I imagine that there are significant costs involved in producing academic journals but I would have expected in the e-book age we might have seen prices beginning to fall for digital versions of articles. After all, if scientists write the papers at no charge to the journal, then other academics peer review those papers at no charge to the journal, can a digital download of that paper really cost that much to produce? Well, perhaps it can, but I do feel these journals have some questions to answer given the important and highly privileged position that many of them occupy.

It seems strange as other sectors in publishing get used to a year-on-year squeeze that one sector is getting accused of profiteering. But then again, there is intense competition in every area of mainstream book retailing, whereas the big names in academic publishing seem to have found something akin to a monopoly.

I think increasingly in the twenty-first century we have to accept that a lot of 'middle man' jobs will go away. Music fans and their favourite bands don't necessarily need a record company in the middle trying to make a profit. Likewise, the science community may find its members can communicate among themselves better without certain publishers being involved. I think the key difference in all these situations will be whether those in the middle work hard to be facilitators, or whether they seem themselves as toll collectors (c.f. the SOPA/PIPA debacle where Hollywood was so fixated on acquiring powers to protect their business model that they managed to provoke millions of ordinary citizens into opposing them.)

P.S. The last time I had occasion to consider the Elsevier name was grumping that their parent, Reed Elsevier, organised UK trade shows for the world's arms manufacturers (link). I believe they have now withdrawn from that line of work.

Well, fancy that. I've found a new command-line way to transform massive XML files. No more java heap space exceptions! No more paying a mahoosive license fee for Oxygen XML just to transform XML!

This is for Mac only. Open the Terminal application. Change into the directory which contains both your XML and XSL files by entering the file path:

cd path/to/your/folder

You can use XSL to transform any sort of XML file. For instance, you might want to convert an ONIX file from its short-tag version to the long reference tag version. Run:

xsltproc shorttolong.xsl short.XML -o long.xml

That syntax is: call the command 'xsltproc'. (The libraries are built in to OSX, so you don't need to have installed anything to get this to work.) Give the name of the XSL file you want to use. Then give the name of the XML file you want to transform. Write -o, and then give the name you want the output file to have.

Simples and usefuls!

I mentioned in my last post that the CDC (that's the Center for Disease Control in America) have issued some guidelines on how to prepare for the coming zombie apocalypse. Of course, they're sneakily telling you lots of things that would also apply if the outbreak were less supernatural in nature. But this is surely more fun. And it seems like they're continuing the pretense by offering zombie posters and now a zombie graphic novella. Posters are here. The novella is here.

poster source

The extent to which I don't have my finger on the common pulse is considerable. For instance, I rarely get my news from TV, which means the current-affairs I know about tends to be skewed towards what seems bad when you think about it rather than on what looks bad when you watch the video of it. But even without all the Mayan calendar nonsense about 2012 being the end of the line, I keep picking up the vibe that lots of people think the end of the world is coming. Soonish.

More and more I read about people obsessing about where they live, where they stash their money or what sort of supplies they keep in the house all considered in the light of Something Bad Happening. And I suspect one of the reasons for the explosion of interest in zombies of late is that it gives us all a chance to consider "what would I do in that situation?". The Walking Dead is a chance for us all to do a bit of personal End of Days planning. And in fact you can find scads of answers to the question of how we should respond to the undead overrunning the Earth from everyone from the CDC to multi-tool maker, Gerber's, Apocalypse Kit.

As well as evangelicals worrying about The Rapture and the rest of us contemplating the Zombie Apocalypse, we've got the real life prospect of climate disaster, we've got the potential catastrophic lock-up of the global banking industry, we've got wars and famines, we've got terrorist threats and we've got periodic panics over pathogens like H1N1 and SARS. And I suppose people can be forgiven for letting those threats seep into the general psyche because - after all - they are real (well, except for terrorism, which statistically speaking, is insignificant in the West)

And among those who don't wish they had a nickel-plated pump-action Mossberg shotgun to fend off the zombies with you can still pick up hints of 'prepping'. Twenty years ago I didn't know many young women who weaved, knitted, grew their own vegetables or made their own clothes. Now the internet is packed with them. In a world where it's easier and cheaper than ever to rely on shops for everything, more and more people are interested in providing for themselves. Handy, if shops ever become a thing of the past.

And given that this is a publishing blog I should mention all those manuals of recent years that teach you how to survive a myriad of life-threatening situations. Why on earth have SAS Survival guides and Worst Case Scenario handbooks sold so well to comfy urbanites?

So what do you reckon? Have you noticed more than the usual amount of armageddon-ism and what I've come to call 'pre-apocalyptic tension'? And if you have, what do you think it all means?

*Both abbreviations refer to 'endtimes': S*** Hits The Fan and Without Rule Of Law

I used to write titles to my blog posts which amused me. But then I could never find them again unless I could remember the joke. So look at what I've been reduced to. It's a blog title composed of utilitarian gruel, I know.

Anyway, what it refers to is a slightly surprising, potentially pivotal statistic. Over Christmas, we went from one fifth of American adults owning a tablet computer to two-fifths. Kapow. Exactly the same thing happened to U.S. e-reader ownership too. Kazap. In fact it actually sounds wrong to me. But I'm assuming that Pew Research know a lot more about these things than I do. And that being the case, it's a big reason why someone should get busy on the whole cross-platform interactive book idea that Apple may have just bungled.

Pew research link.

Just to wrap up my recent (1 and 2) focus on the new iBooks 2.0 interactive textbook launch there are two snippets to add.
You might not have to upgrade to the latest version of the Mac OS to use iBooks Author after all. Supposedly it's for Lion only, but tech-minded tinkerers seem to have got it to work with Snow Leopard. See here for the fiddly details.
Apparently the EULA that comes with iBooks Author is the most greedy, grabby agreement of its kind according to someone who examines them for a living. See here. Apparently it attempts to control what you can do with the documents you create. People have likened it to trying to control what you can do with a Word document you write.

All in all, Apple seem to have taken a great idea - bring the world interactive textbooks - and done a slightly sleazy, short-sighted and tight-fisted job of it.

Imagining a new way of learning and then seeing someone build it (and show it off in a slick, glossy video) is a very exciting experience (as I mentioned in my last post). Last night I did a bit more reading about the iBooks 2.0 format on which this new Apple interactive textbook platform is based and I'm now 30% less enthused. Maybe as much as 60% less. It's difficult to get a really accurate figure until the field of neuroscience advances a little, but suffice it to say the latest weather forecast for the iBooks parade now includes rain.

When I started getting excited about iBooks Author, the program Apple has just released for creating these interactive textbooks, it was largely because I thought they were building an ecosystem in which the textbook industry could make their new home. And perhaps they will. But this first version appears to be a closed system: you use an Apple program to make a book you can only read through Apple's iBooks software and can only sell through its iBooks store. And you can't easily carry over the work you've put into that book into a version that works elsewhere because you can't export your designs into other, more open formats. And since the iBooks 2.0 format is chock full of undocumented features it's going to be very difficult for anyone to write a conversion tool. Sad face.

The bad news begins with what Apple have done to ePub3. Because in its standard form it didn't allow the functionality they wanted, they extended it and in doing so added lots of their own unorthodox syntax - syntax that would not just be ignored by ePub readers but would almost certainly trip them up. Apple also opted to achieve lots of the clever animations and interactivity using little programs, called 'gadgets', that they wrote themselves and which are basically black boxes as far as the rest of the industry is concerned.

The neighbourly way to add cool new features to web sites (and so logically the way to do the same in a format based on web standards) would have been to write those gadgets in JavaScript. That's how Google Maps and GMail do all sorts of things that no web page ever did before. But iBooks 2.0 doesn't go down that route. It adds in lots of custom-built modules that only Apple understand and controls them with lots of custom-built syntax that Apple haven't documented.

I have to say, based on these revelations, this first iteration of iBooks textbooks is sounding like a very beautiful dead end. I'm particularly disappointed that the EULA apparently prevents you selling the books you create with iBooks Author anywhere other than the iBooks store. It all seems a little short-sighted to me. Given that most people will want the option to read textbooks on something hand-held rather than a laptop or desktop, it seems to me that this is all about iPads. iPads are fabulous and they've sold very well, but I can't see most publishers committing to them so thoroughly and exclusively at this stage.

It's early days yet, but this seems like a mis-step to me. This is the sort of behaviour we used to get from the bad old Microsoft who were always trying to slap the handcuffs on their customers to stop them wandering away. And Microsoft too were insouciant to an extent that seemed like actual contempt towards open standards that the rest of the technical world had painstakingly built. We're still living with the legacy of Internet Explorer which seemed designed to lock its users into Microsoft products by locking out any regions of the web that were based purely on internationally agreed standards. It's a shame if Apple are now trying the same thing.

Apple clearly had to go beyond existing standards if their interactive textbooks were to do all the cool things Apple wanted, and it's great that they began their work from the base of ePub3. But they appear to have built a cramped little 'walled garden' where you can't get outside content in and you can't get inside content out. You have to use an Apple product to put together an Apple book and then you have to read it on an Apple device. That sounds like far too much 'lock in' for a new platform. And unless they start building some doors into their walled garden I don't see iBooks textbooks being anything other than a beautiful novelty.

Read more analysis here at Baldur Bjarnason's website and check out the links at the bottom of the page for lots more detail from him and others.

P.S. I've been rather harsh and dismissive about the fact that Adobe haven't provided decent e-book creation tools yet - despite making the dominant DTP app - and I continued to be critical of them when I first saw iBooks Author because it looked like they were allowing Apple to set about eating their lunch on a persistent and ongoing basis. But perhaps Adobe still have a chance. If they're frantically working on something that helps the majority of the publishing industry produce this and the next generation of e-books then they might still be in with a chance. And I'm not talking about a clunky add-in to InDesign like their current e-book export fudge.

P.P.S. Amazon have just released the first round of tools to allow the outer tier of publishers (i.e. those not on best buddy terms with Amazon) to start producing KF8 e-books that will run on their Fire tablet and presumably elsewhere. When I get a minute I'm going to take a look at those tools and report back on what I find. Take a look for yourself if you haven't already. link

For a while now I've been mentally preparing for a world where non-fiction books could go digital and could contain interactive, intelligent, visually responsive content. Last year I talked about the possibilities here. If Adobe were on the ball, they'd have created a platform for these things, but they seem to be letting the world of e-books pass them by. Unsurprisingly, Apple have stepped into the breach and produced a free authoring program for creating interactive textbooks. You can go here and see their video about the ideas behind this new technology and what it's already capable of. Yes, the video is full of treacle and meaningful shots of multi-ethnic kids smiling, but I defy you to not get a little bit excited about the potential here. I've already downloaded a free copy of iBooks Author and when I get a chance I'm going to properly explore it. And remember, this is Version 1. Apple have a strong history of building on their innovations until they really create something powerful.

Downsides: to run iBooks Author you'll need a Mac. And it'll need to be running Lion, the latest version of the OS. Other downsides: if you work in non-fiction publishing you've suddenly got a lot more work to do.

update: check out my follow-up post with what I consider is some solidly bad news about the iBooks 2.0 platform.

I'm quoting from an article at Pando Daily where Sarah Lacy interviews an anonymous, but nevertheless quoteworthy publisher:
"Long-term there’s no future in printed books. They’ll be like vinyl: pricey and for collectors only. 95% of people will read digitally. Everybody in publishing knows this but most are in denial about it because moving to becoming a digital company means laying off like 40% of our staffs. And the barriers to entry fall, too. We simply don’t want to think about it. Amazon is thinking about it, though, and they’re targeting the publishers directly."

And: "To be honest, publishing is a quaint little industry based on romance and low profit margins. But now we’re in Amazon’s sights, and they’re going to kill us."

Provocative and polemic. Over the top, maybe. But I did have a brief moment while reading the article where I thought, "of course. this is all true." Probably because it's just a flashy and slightly paranoid version of what's been swirling around in my brain for a while now.

Did you watch the second season of the BBC's Sherlock? You might just about be able to catch all three movie-length episodes on iPlayer if you're quick. I'm all conflicted about it. The first ep was absolutely wonderful. The second episode disappointed me. The third episode left me fuming.

Comprehensive spoilers ahead.

Steven Moffat on a good day creates fiendishly complex plots which actually make sense with hindsight and he packs them with humour. On a really good day he also gets the drama right. I found I had a piece of dust in my eye at the very end of A Scandal in Belgravia, when Irene Adler faces the chop. I watched the episode again a few days later and found the same piece of dust once again troubling me. Given what a macho brute I am, and how fiercely out of touch with my emotions I am, it's pretty impressive for what is ostensibly a funny intellectual puzzle to move me, even as a 'repeat'.

I've said it before, but Mark Gatiss seems like a lovely fellow. He's charming, funny and intelligent, and he seems like a pretty decent actor too. But I have yet to enjoy his writing. With s02e01, Moffat gave us riddles to solve against the clock while our heroes had a gun to their heads. in s02e02, Gatiss gave us what amounted to dream sequences. And once I'd worked out that people were hallucinating, the tension evaporated and my interest along with it. Ep 1 had thrills; ep 2 had scares: scares of the sudden-loud-noise or monster-jumping-out kind. We've all seen a hundred million of those and they're not traditionally what we demand of a detective show.

And with Sherlock of all shows you can't use the excuse "it's only TV; you're not supposed to think about it too much."

But it's s02ep3, The Reichenbach Fall, that was in danger of undoing all the good work that Scandal in Belgravia had done in making me a giant fan of the series. Detective shows are about solving puzzles. Half the fun is trying to beat the detective to the answer. Scandal in Belgravia delivered in abundance. On the other hand, the Reichenbach Fall was full of puzzles that were later dismissed, puzzles where the answer was the first thing anyone would think of, puzzles where the answer didn't make sense and puzzles which were simply not answered, or even addressed, presumably to be picked up in 12 months' time when we can hardly remember what the question was.

In s02e01, we had puzzles like 'how was the hiker killed', 'what's the combination to the safe?', 'what's the password to the phone?' and 'why wasn't the dead man on the flight?' We were given hints and partial clues - enough to solve the riddles - or at least to have the solutions make you slap your palm to your forehead - all the while being dazzled by the dialogue and swept up in the drama. What was Irene Adler's game and were she and Sherlock really engaged in some sort of romantic dance or was it all a smokescreen? Every one of these intellectual or emotional questions was paid off in a satisfying, intelligent and frequently exhilarating way.

In s02e03 we had the questions of 'what does i.o.u. mean?', 'what is the final problem moriarty refers to?', 'what are the professional killers up to?', 'where are the kidnapped children being held?', 'how did moriarty pull off those break ins?', 'how did moriarty get acquitted?', 'how did moriarty create a fake persona as a TV presenter?' and 'where was the secret key hidden?'. The answers to the first two were simply not addressed. The answer to the question of the killers... just didn't make sense. They were told to 'get Sherlock' which they apparently interpreted to mean saving his life and bumping each other off (but only when one of them touched Sherlock) until... they could work out through lots of quiet contemplation where he'd stashed the key, presumably? And they were working for themselves hunting for the non-existent key until Moriarty wanted them to become a threat at which point they were to be considered utterly dedicated to killing Sherlock's friends unless called off by Moriarty. Couldn't Sherlock have offered them the (non-existent) key to bribe them to let his pals live? Apparently not. Or were the hired killers ready to bump off Sherlock's friends a different group from the hired killers protecting him? Who knows. Killers, killers everywhere and not a drop of sense.

The answer to where the children were imprisoned involved some nineteenth century chemistry performed upon the residue dissolved in the footprints from Moriarty's shoe. The business with the linseed oil seemed to be a failed attempt by the kidnapped boy to lead the police to his abductors, but it had the (presumably unintended) effect of allowing Sherlock to identify all the traces on Moriarty's shoe. And the implication was that Moriarty knew Sherlock would pursue this approach and was even hinting that he had left a breadcrumb trail. But wasn't the linseed oil the boy's idea not Moriarty's? All very confusing. And all very unconvincing. Not to mention technically hokey. Steve Thompson (= the writer) has clearly never seen CSI or read a book on forensics: I could almost believe a mass-spectrometer could identify invisible microscopic residue, but I just can't buy the bubbling test tubes and litmus paper approach.

And more importantly when considering that this is a detective show, these weren't answers you could work out given the clues and which, with hindsight, seemed impressively logical; they were ad-hoc nonsense.

The answers to the questions about the robberies, the acquittal and presumably also the fake persona (though this also was never addressed) was even more of a let-down. Moriarty simply threatened or bribed a few key people. Just the sort of thing the police, even without any help from Sherlock, would normally suspect, and which they would presumably uncover fairly quickly. As intriguing mysteries go, this is akin to discovering that a burglar got into a house by breaking a window. Or was it a sneaky double-bluff: you thought this was an interesting puzzle but really it was boring. If so, I'm not impressed.

And as for the secret key, it never existed. So Moriarty had the entire underworld bidding for his services knowing that he couldn't deliver. Or could only deliver to the extent that ordinary bribery and threats could achieve. Presumably this was a cunning plan on Moriarty's part to anger every powerful criminal in the world and to make himself look like a fraud. How accidentally ironic, given that Moriarty was so intent on making Sherlock appear to be a fraud. And the court case would have sealed his fate by letting all these potential criminal employers know what he looked like. Or would the world's crime lords suddenly decide that Moriarty was really a kid's TV presenter because The Sun said so? Who gets the world's rogue states bidding for their time and then announces that really they work for CBeebies? It's probably a good job Moriarty shot himself because his days were presumably numbered.

Which leads me to the final plot point I absolutely do not buy. It makes no sense that Moriarty was desperate to destroy Sherlock but wouldn't bother to stick around to see it happen. It wasn't enough to discredit Sherlock; Moriarty needed him to admit defeat and take his own life. If that final submission was so important, why blow your brains out on a whim and miss it? Moreover the reasoning for Moriarty taking his own life appeared to be that Sherlock would be able to make him call off the assassins otherwise. Moriarty had withstood weeks of torture by professional intelligence types lead by Mycroft (who in the books was considered even more intelligent than Sherlock). But apparently on the basis of some really solid eye contact, Moriarty conceded that Sherlock could swiftly break him, and so quickly committed suicide.

The possibility that Moriarty too was faking his own death doesn't make that whole plot point easier to digest. It just makes a mockery of the whole showdown by having both participants fake their own suicides in quick succession.

The one thing which might have partially salvaged The Reichenbach Fall for me would have been appending the words "To Be Continued" to the closing scene. At least then I would have felt all the ridiculous loose ends and nonsensical plot points were being acknowledged. I would still consider that a movie-length episode is enough time to wrap up a story. I would also consider it a tactical miscalculation to build dramatic tension for two hours and then offer to resolve the whole thing in a year's time. But they didn't even do that. They simply ended the episode.

I suppose I should concede that the question of how Sherlock faked his own death might prove to be a satisfying riddle suitably in keeping with the spirit of the show. On the other hand, we've got 12 months or so to work out the answer and I suspect that the internet has already assembled all the important pieces of the puzzle. By the time Christmas 2012 rolls around we're not going to be wowed by a pile of bin bags, a corpse borrowed from Molly's mortuary and a whiff of the HOUND drug from ep 2 for Watson. Plus I'm already suspicious that the reason Watson couldn't be in on the deception will prove to be 'because otherwise the killers wouldn't have been convinced' - which, if that's the explanation given, I'm not going to buy. A bit of blubbing at the funeral and most professional killers would be satisfied I reckon.

And finally, I don't see what the finale episode achieves for the series as a whole except to set up a lot of annoying obstacles to good storytelling next year. Sherlock presumably can't easily go back to being a detective because he is now famous for being a) a fraud and b) dead. He can't go back to working with the police given the appalling press the police will have received for inviting in a charlatan who rigged dozens of their cases. Or will an episode of next season be spent erasing all those obstacles rather than letting us have intriguing crimes to solve?

If you'd asked me about Sherlock after s02e01 I'd have said the show is one of the best things I've seen on TV. Having seen eps 2 and especially 3, I'm not sure what to make of it all. Clearly they're capable of amazing things. I'll have my fingers crossed that they manage something amazing again - at least once - next season.

Today great chunks of the internet are offline as a protest against the SOPA and PIPA legislation that the U.S. Congress is soon to vote on. Like many pieces of anti-piracy legislation, it seems very happy to throw a lot of babies out with what might be a very small amount of bathwater. It contains a lot of bad ideas that have been proposed (and even partly adopted) before and will be again.

The heart of the debate is "how far will we go to defend the interests of copyright holders?" If you're an author or a publisher you probably have a dog in this race. You probably don't want to make it easy for people to use your copyrighted works without your permission. And when I've mentioned this subject before I've got the impression from the comments that many visitors here are in favour of very strong protection of copyright. For instance, when I've complained about the works of dead authors being protected for fifty or seventy years after their death, some commenters have questioned whether those protections should ever expire. I mean, why should what amounts to your personal property ever pass into public ownership?

I think, though, we have to accept that works shouldn't be 100% protected. For instance, imagine you could copyright a really well turned phrase and collect royalties whenever anyone wished to use it. Perhaps you spent a whole year coming up with it and feel you deserve remuneration when anyone threatens to pirate it. If protection like that had existed since, say, 1800, imagine how stilted the typical novel would now be as it attempted not to pay out a fortune in well-turned-phrase royalties. Likewise if you could copyright the idea of a detective novel or a happy ending there'd only be a handful of books in most of today's more popular genres. A society that offered protections like that to authors would make a handful of friends and a vast number of enemies amongst its citizens. It would also spend a lot of money protecting the few against the majority.

Somewhere along the line it became obvious that what is good for an author is not always good for authors in general or society as a whole. We don't want to protect one dead author's works at the expense of every writer who comes afterwards who wants to do something similar - and at the expense of all the readers who don't want just one version of a story, they want a whole section of the book store exploring that theme.

So societies weigh the cost of protecting an author's work against the expense of that protection and the damage that protection will do elsewhere. Without any protection, perhaps all authors would stop writing (though I highly doubt it). But too much protection (as mentioned above) can throttle creativity even more surely than too little. After all, more people will write for free than will write with the threat of litigation hanging over them.

In other words, protecting authors incentivises them to write (because it permits the possibility of making a living that way) but limiting that protection keeps the playing field open and level, and allows writing as a whole to flourish. Limiting protection also limits the burden on society of enforcing copyright.

For scientific developments the balance is slightly shifted. If a medicine comes off patent, we can all benefit from it, but without patents there's no way to make back the huge R&D costs of developing new treatments, so society attempts to grant just enough protection to encourage future development before it turns new ideas over to the public domain. And don't forget, for many patents it's a race: lots of people are working on the same idea, but society chooses to reward only the first to finish. That's not about encouraging innovation so much as encouraging speed.

And then there's the question of all the 'grandfathering' that goes on. You can get into trouble for printing unlicenced copies of a Lichtenstein or Warhol print. But those artists made their names by freely stealing and reusing other people's artwork without permission. And for all that Justin Bieber's music company wants heavyweight copyright protection, Justin himself got started making YouTube videos of other people's songs. And as the very funny TV show Community pointed out: if the schoolkids from Glee tried to sing those songs in real life, they'd be asked to pay big money for them or they'd be shut down.

Many (most?) artists have made free use of every work that came before them but then they, or more likely their management companies, try to ban anyone else from doing the same. Or at least companies insist that creators of derivative works must pay for the privilege in a way that they themselves didn't. The authors of the SOPA and PIPA legislation seem to consider every YouTube homage, mashup or re-enactment as piracy but they also want to benefit from buzz and viral marketing. They want everyone to obsess over their new movie, just not in any way they might be able to charge you for.

As Naomi Klein pointed out in No Logo, Disney wants your child to have a Lion King backpack, pencil case, lunchbox, video game, colouring book, duvet cover and wall poster. But they also want such tight control over their brand that you're not allowed to hang that child's drawing of Simba on a classroom wall. I mean *of course* Disney want that. The question is whether society as a whole wants to put their effort into granting and enforcing that level of control.

To turn to SOPA and PIPA specifically, they attempt to enforce far-reaching copyright control by altering the technical and also the legal underpinnings of the internet. On the legal side they want copyright holders to be able to have websites taken down immediately if they are hosting pirated content. Which sounds reasonable. But 'immediately' means without having to go through a tedious court case - or even showing any evidence. It is enough simply to accuse. And there are no penalties for making a wrongful accusation. Moreover 'immediately' also means shutting down the server on which the (possibly) pirated content is hosted - even if it means taking down a dozen other websites too.

That would be bad enough if we could trust copyright holders to be scrupulously honest, but under the current U.S. legislation (the Digital Millenium Copyright Act - DMCA) 'take down' notices are often used to silence critics rather than to prevent piracy.

Worse still, the plan is to make the hosting company responsible for whatever their customers put on their servers. So if you upload something to YouTube that infringes copyright it's not just you who are breaking the law, it's YouTube. Once one has factored in the cost of a copyright lawyer spending a day on each YouTube video attempting to research the relevant intellectual property ownership it becomes pretty clear that YouTube would no longer be viable. Which not-exactly-incidentally would be a boon for the TV companies insisting on SOPA/PIPA - by shutting down their competition in the name of piracy-protection they could receive a commercial boost too by decimating the world of user-generated content.

And that's even supposing it were possible to recognise every scrap of copyrighted video and every bar of a protected melody. Depending on one's interpretation of this legislation Google would also have to go - because hosting a link to pirated material is also punishable. And for the same reason Facebook and Twitter would be under threat.

In fact if you wanted to get, say, the New York Times website taken off the internet you'd simply have to post a few links in the comments to harmless looking sites, which actually offered pirated copies of Dumb and Dumber, and then issue a 'take down' notice for repeated copyright violation. The only sensible policy in such a world is to drastically limit file hosting and the posting of comments as well as the inclusion of links to anything you don't control.

Of course those demanding this level of protection (mainly the Hollywood and music industries' lobbyists) constantly fall foul of it themselves - not that they care. Illegal downloads of movies and music have been traced to computers in Congress, but presumably Congress isn't going to ban itself from the internet. And Lamar Smith, one of the Congressmen behind the SOPA legislation would have had his website taken down under the new legislation. He used a picture of some foliage as a backdrop on his site, but he didn't credit the picture's original owner as required by the photo's licence (link). If this becomes all it takes to silence someone online, the internet could get very quiet very quickly.

And that's just the legal side. SOPA and PIPA also mandate some re-engineering of the internet's infrastructure. When you type a website name into your browser, your computer turns that into a string of numbers which, like the internet equivalent of a phone number, allows you to contact that website and fetch pages from it. These numbers are called IP addresses and the service that allows web names (like snowbooks.com) to be turned into IP addresses (like 74.86.153.134) is called DNS. It acts like a phone directory and SOPA/PIPA would give enforcement agencies the power to edit those DNS directories (which are spread all over the web) and to delete entries that were deemed undesirable. By removing a website from DNS you'd make it impossible to find. Except that geeks have already found ways round that - but SOPA/PIPA in turn gets round that problem by making it illegal to circumvent the deletion of DNS entries.

Now in countries like North Korea and Syria and China there are similar rules, but they are used principally to silence dissent, and the U.S. State Department is wholeheartedly against them. But SOPA/PIPA would introduce the same mechanisms of online information control (albeit serving corporate not dictatorial ends) and would ban the creation of any tools to circumvent that control. In other words, you wouldn't be able to find ways round internet censorship in China because that same technology might also be used to download an illegal copy of Deuce Bigalow: Male Gigolo.

Moreover, building mechanisms that could be used for censorship into what has traditional been a free internet is not a development that pleases advocates of free speech. America effectively controls much of the internet because it originated much of the technology and because it has had the good sense to keep its hands off the controls. If Congress builds censorship technology into the internet it stands to lose America its position as the notional headquarters of the web - which might not be a bad thing in the long run, but it's almost certainly not what most of America wants.

Even if SOPA and PIPA are defeated, this is not a debate that will go away. On the internet it's ridiculously easy to copy everything: ideas, movies, music, schematics, blueprints, personal information and leaked e-mails. And it's going to keep getting easier as computers, software and networks get more powerful. But those who stand to have their business models swept away will want to take steps to prevent the copying that threatens them. Sometimes they will have good ideas about how to do that. Sometimes they will just want to be put in charge of everything so that they can ensure no one is misbehaving - and when that happens they need to be stopped. But it's not a problem that is going to evaporate.

If you want to know more about where this debate is going, I wholeheartedly recommend watching Cory Doctorow's talk on The Coming War on General Computation (link). It's really worth listening to his explanation of why this is such an important and difficult problem, but to give you a taste, he explains how we keep trying to ban people using technologies in ways we don't like by assuming it's like banning mobile phone use in cars: you can ban it and people can still use their cars. But limiting piracy on the internet is more like trying to build a car that can't be used in a bank robbery but can be used in any other way - and that's a much more difficult problem. And as Cory points out, even if you're against strong copyright controls for books and movies, you might find that you're in favour of censorship when it comes to downloading and printing out copies of the smallpox virus, for instance.

So. Lots to ponder. But in the meantime, I hope Congress will bury SOPA and PIPA, that they'll give at least equal weight to their voters as to their corporate lobbyists, and that next time they'll read up on what they're planning to re-engineer.

Addendum: Also, here's Clay Shirky giving a quick talk at the TED offices about the background to SOPA/PIPA as well as its implications. He's firmly against the legislation, so he's a little one-sided perhaps - that said, I think he's being accurate in what he says, plus I happen to agree with him. And he does a fun job of bringing us all up to speed: link

It's become a bit of a cliché to wonder what an Apple version of something would be like - assuming it doesn't already exist. The firm totally redefined entire industries with their mobile phone and then their tablet, and there's talking of them tackling TVs soon. But an announcement that's crept up on me a little bit is that they've also been working in the field of interactive books - specifically tools for creating them. The sound bite seems to be 'garageband for e-books' - which won't mean anything to you if you don't know what GarageBand is (it's a music-making program with a famously intuitive and non-technical interface). On the 19th they're going to unveil whatever they've been working on and a lot of people think it'll revolutionise the textbook and drag it into the electronic age. More details here.

Well, that's a chirpy title for an article, isn't it? It's written by Trey Ratcliff of the travel photography blog StuckInCustoms. He's also the founder of Flat Books. The article contrasts how you sell a pbook* and the kind of money you make, set against the evolving practice and profitability of selling ebooks, and it's based on his recent first-hand experiences. My summary: e-books can be fat moneyspinners and they're only going to get better. So head over and have a read if you want to be buoyed up about your e-book prospects. (He also bandies around terms like 'doomed' and 'dying' when referring to pbooks so depending on what you do for a living the net effect of the article on your mood might be down instead of up.)

*It's probably annoyingly faddy and jargonistic of me to refer to paper books as pbooks. On the other hand, 'paper books' sounds stilted and I'm tired of calling them 'traditional books'. But when I'm talking about e-books I need a way to convey the idea of non-e-books that's not too clunky. Any better suggestions?