Back to the drawing board?
Right2water’s IT professional says communication and openness is key to mutually acceptable campaign software solution.
We sat down with Xavier Dutoit, Right2water’s IT specialist, who shared with us some of the trials, tribulations and unexpected outcomes of signature collection. His observations and insights will be of particular interest to future ECI organizers.
You are the first IT expert to conduct a successful ECI using the Commission’s (EC) Online Collection System (OCS). Congratulations! What are your experiences with this OCS?
In one word, my experience was unsatisfactory. The situation was highly problematic. The main issue was that we were a campaign and needed a campaigning tool but the OCS software developed by the EC to collect signatures was oriented towards administrative goals. And there’s a big difference between these two aims. At our end we wanted to have a successful campaign but the signature collection was not entirely in our hands.
There were several problems during our long year of campaigning, many of which happened as a result of a faulty validation procedure. French signatures, for instance, were not recognized because the formats were unsuited for French passport and ID data. This issue was partly the responsibility of member states that had not submitted all of the existing valid formats to the EC. And we had the +that same problem in a lot of countries. The impression I am left with is that the OCS is designed to reject as many signatures as it can. If there is the slightest doubt then the system will reject it. Every time the software could block us it did. We really felt as if the software was not working in the interest of campaigners.
We also learned that you had signature losses that were a direct result of the Commission’s OCS. Did this only happen in the beginning or over the course of the entire campaign as well?
In was clear from the very beginning that the OCS did not work. We actually had expected this (see also Xavier’s article published before the launch of r2w: “ECI, or how “security theater” is excluding Civil Society”). The OCS was rejecting valid national ID formats, and then we had other kinds of problems with the software itself. Keep in mind we were the first ones using the software and we were hosting on our own server while collecting signatures online. So we were beta-testing and debugging live the various problems which we faced. We also had problems with the fact that it couldn’t handle the load. It worked fine when we had a few hundred signatures per day, but then it crashed non-stop for more than a week when we started getting traction.
And obviously when you are campaigning and you start seeing that people who tried to sign are finding out they were denied the possibility it meant that they would tell their friends that this ECI “doesn’t work”. So not only are you losing signatures of the person who tried to sign but also the potential of a snowball effect to promote and reach out to friends and friends of friends…
Since the OCS was not designed to be a campaign-friendly tool all the experience over the past 15 years about how to make it easy for people to take action were not incorporated. As for the software it was difficult to use. We had probably tens of thousands of users who contacted us saying “”we can’t sign because we’re not able to fill in the captcha” – with the little image where you have to retype a code – because it was too complicated. That is something that we and the other ECI organizers had been saying to the Commission from the beginning.
At the ECI seminar in Brussels earlier this year (March 19th) even the EC civil servants from the Secretarial General admitted they had tried three times to sign. Did anything change after this?
Yes, even people within the EC from other DGs publicly stated that this needed to be fixed, but it didn’t happen. At that meeting Mario Tenreiro (from the “political” side) asked if it was possible to make it easier to read. Francisco Gimeno said it was, and was asked to fix it.
Since that time two new versions of the software have been released, but the captcha makes it as difficult to sign as ever. Meanwhile, the Commission states that “among the usability improvements signatories are now required to fill in the form in capital letters”. Out of nearly 2 million citizens, no one was asked to have their signature blocked unless they enter it in capital letters, but clearly the EC deemed it more useful to block a name typed in lowercase letters than to fix the captcha.
What were your biggest challenges and difficulties with the official OCS, in a nutshell?
What we had been saying to the EC is that the main focus regarding the OCS should be integrating it with the campaign website. Because right now we technically have two different sites: one that is trying to engage the user and get multiple thirds and other serving as a fortress that’s as difficult as possible to enter. And it wasn’t possible to link easily between these two elements of the campaign – the secure mailbox where the signatures are collected and the campaign website. That was the first major issue we had.
The second issue was design and layout. The software is not engaging. It doesn’t look nice. It doesn’t look like it belongs to a campaign and isn’t suited for one.
The third was about the process. The EC doesn’t have any experience with campaigning and doesn’t have any experience developing open source software. And one of the key things is you have to be open. Neither of us knows everything that needs to be known about how to offer platforms and run online campaigns. We know a few tricks for a decade or more of experience. But we don’t know everything. So the first thing is we say: I am working on a new feature. This is the test version. Go try it and let us know what you think about it. When we said this their response was to create two different committees with workshops and meetings, etc. – which is not the way you actually do open source development. That’s not how you get the answers and assurances you need. Keeping the campaigners informed of the dialogue between the organizers and the EC developers either did not occur at all or didn’t occur in an efficient way.
The EC responded to the difficulties firstly by extended the deadline for every registered ECI, and secondly by offered to help through the Luxembourg authorities. How far did this solve the problems?
It seems the Luxemburg authorities were helpful for other ECI organizers. For us it didn’t have much of an impact since we hosted the software on our own server. So we already went through the problem of certifying the server and software, and so on. So providing the hosting didn’t help. But the duration of one year for signature collection was extended.
Right now the EC can decide when exactly an ECI is being launched. They have up to 2 months to decide when a registered ECI can start. For organizers that means you don’t know when exactly you can launch a campaign. You don’t know when the clock starts ticking. The ECI organizers should have the right to choose exactly when to launch the ECI once it’s admissible.
Is it helpful to use one’s own server?
If you don’t have your own server you are losing visibility of what’s happening on the server. It was not easy to maintain one server but we could see what is going on and we could see how many signatures we were losing. We could see all the problems we had. And that’s information that you don’t have if you don’t have control and don’t have access to the server. Discussions with other ECI organizers show that they are dealing with an even bigger black box compared to what we had. And it’s even harder for them to know how many people are visiting and how many people click something, what is happening when people stop in the middle of the process and how many error messages are there at this point. This is information that’s needed in order to run a good campaign.
It seems we can no longer expect too much from the institutions. I recall that at the April conference at the EESC on “ECI Day” that you were on a panel with the EC responsible for the online collection system (Franciso Gimeno). The EC said, essentially, that they had done everything they could. How true is this, if the users apparently are not at all happy? And what are your hopes, and practical suggestions?
Well, I had several meetings with the EC. The process and analyses was slow – much too slow for what we expect as campaigners. Not too long ago we installed the latest version that was released and there is very little improvement even though it was under development for an entire year! Even the number one of complaint – simplifying the captcha – wasn’t addressed in this version.
I think what should be done is to start from scratch. We know what the needs of campaigners are, we are familiar with the regulation and we know how to meet its specifications. Let’s develop software that is more efficient and more campaign-oriented. Because now bureaucracy is preventing us from getting signatures. And we need to turn the table around and say: the goal is to engage and the software should work towards that goal and not against that goal. And I think now it is high time to face the issue.
We’ve seen that it’s possible to run a successful ECI, we’ve also seen that the software has a tremendous part to play in this. It’s extremely difficult to run an ECI as it is but the software makes it even harder. We’ve seen that a few improvements could make a big difference, but let’s make a fresh start. Let’s design the kind of software that campaigners want and need to be able to conduct a proper ECI. Up until today we’ve received weekly complaints from users who have problems with the captcha. We have had tens of thousands of complaints like that.
What kind of behaviour were you able extract and identify based on your statistical data?
What we observed with the latest software version is that the person that signs the ECI wants to be informed about the campaign and the result of the ECI. Anyone who has run a campaign views this as logical. Obviously if someone takes an action they will want to see the result of that action. But the software was designed in an extremely restrictive way: once a person signs, the interaction is over. You don’t have any opportunity to register for a newsletter. And legally we don’t have the right to use the data of the supporter. Once you sign you give your name, ID number and address, etc., the only thing we can do is to give that information to the member state for validation. Then we have to delete it and can’t even use it to let the person know that the ECI was successful. Due to the software design we lost an opportunity to contact some 150,000 people. We could have easily collected 1 million e-mail contacts (from those who would want to be contacted).
Now on the bottom left-hand side of the screen you have a little link offering the possibility to “go back to the campaign website”. What we did here is that we arranged it so that when the supporters click on this button they go back to a page where they can register for the newsletter so we can inform them. Still, it doesn’t take place in a very user-friendly way because you have too many clicks and steps in between. But even with all this fuss we have nearly ten thousand who both signed the ECI and registered for a newsletter.
What results do you have on website traffic during the campaign?
Besides the number of signatures we did go back to the traffic on the main site (keeping in mind some visitors went directly to the signature website and aren’t counted) and could identify that over the year there were 6,607,832 page views and 3,611,920 visitors. On the 24th of January alone we had 362,583 page views and 182,781 visitors!
Are there any other specifics you want to mention, or lessons learned?
Something I find surprising is how little traffic came from partners and affiliates. Combined these represented less than 1% of the total traffic. Visitors won’t come directly from partner sites, but mostly come through newsletters and social networks. That said, it’s still super important for page rank. Tablet and smartphone use account for about 10% of the visitors – this could have been much higher.
The impression of working at cross purposes with the Commission appears to be one of the most commonly expressed grievance of the first bold social reformers attempting transnational democracy – and with an instrument which is itself still in an experimental phase. As guinea pigs for the EC software, we can be grateful to Xavier and other organizers for attempting to work out the kinks in the technology while trying to achieve their objectives. We can applaud the Commission, too, for its pioneering efforts trying to engineer such a complex new instrument. We can only hope that in future we will see a Commission that perceives itself as a partner in this experiment, open to collaborative work with people who possess the right skills for the task and willing to surrender some of their own priorities to the needs of citizen organizers.
The interview was conducted by Carsten Berg, Coordinator of the ECI Campaign.