TID technical specifications

TID+ Technical specifications

Grant agreement no. eParticipation 2006/01/21 Project TID+ Deliverable no. 7

November 11th, 2007

Introduction ............................................................................................................................................... 3 Features ...................................................................................................................................................... 4 Connections to outside discussions ....................................................................................................... 4 Simplified idea process, adjustable quorum level ................................................................................. 6 Idea pre-selection .................................................................................................................................. 7 Rejected ideas, reasons for rejections ................................................................................................... 8 Categorization / tagging system ...........................................................................................................10 Idea recycling .......................................................................................................................................11 User authentication ..............................................................................................................................12 Reputation and self-moderation system ...............................................................................................13 Trackability ..........................................................................................................................................14 Software related requirements ..................................................................................................................16 License .................................................................................................................................................16 Internationalization ..............................................................................................................................16 Skinnability ..........................................................................................................................................17 Easy installation ...................................................................................................................................17 Scalability ............................................................................................................................................18 Privacy with accountability ..................................................................................................................18 Search engine optimization and usability ............................................................................................18

2

TID+ Technical specifications Introduction These technical specifications for transforming TOM into TID+ can be divided into two broad types: base-software-related requirements - that must be implemented to make the internationalization and distribution of TID+ possible - and features that address the user needs described in "TID procedural recommendations".

As the first group is mostly technical in nature and perhaps less interesting for the reader, this document starts with features that we believe cannot be achieved by simple procedural changes but need additional software development. Features are accompanied by an illustration of possible implementation, which is not to be considered an exacting requirement but an optional example. Also, as the development is followed by a testing phase it is entirely possible that some of the features are abandoned in case they do not in fact help to improve the system or turn out to be too difficult to implement in a usable way.

3

Features Connections to outside discussions Problem: all significant increases in TOM usage are related to a specific idea being covered in media, forums, blogs etc. Unfortunately this outside interest is not visible on TOM and TOM content is not syndicated in a way that would make including this outside publicity easy.

Both social and mainstream media have become accustomed to using certain tools for this kind of cooperation; so as to participate in conversation TID+ needs to support the same technologies. Most of them are easy to implement as additions to the webpage code or customizing RSS output possibilities, actual effectiveness (and inclusion in distribution) needs testing.

•

automatic tracking via technocrati.com (link to search of the idea’s URL, if possible then Technorati API to poll for number of references)

•

manually adding links to outside coverage (admins, authors or all logged in users?)

•

trackbacks and incoming referrers (needs spam protection solution – or simple solution like keeping referrers visible only to admins/authors)

•

solution to allow 3rd party websites (.gov, media) to include latest TID+ ideas on their pages (either all or specific tag / subject group)

Technorati.com is a blog search engine tracking over 100 million weblogs making it possible to search for keywords or URLs. As technorati.com periodically checks RSS feeds and follows „pings“ from weblog software it indexes content quicker than traditional search engines that pay more attention to relevancy. A sample of this usage can be seen on http://commentisfree.guardian.co.uk (green talk-balloon after number of comments).

4

Other forms of promotion that are often grouped together include making it easy to save TOM links to del.icio.us or Digg.com.

Sample: links to external discussions and possibility to save / tag link under Steve Rubel's micropersuasion.com post.

Adding links to blog posts as one form of comments is often available in blogging software and has been used by communities like photo.net for years. As this option may be used for spamming it is important to limit it to registered users and provide some means to see recently added links in one central location so that administrators can remove possible spam.

Weblog software uses „trackback“ calls: when user A publishes a post that links to user B’s posting A’s server notifies B’s server so it can list a link to A’s post as a comment – an easy way to allow discussions that span multiple sites. As this requires the implementation of XML-RPC call (and would make yet another form of spam possible) another option might be to make the referrer log visible to logged-in users (when a user clicks on a hyperlink the target server is also provided with the URL of originating page, thus making it possible to notice when a particular source such as an online article or a forum posting starts to send traffic).

For promoting TID+ ideas on other sites options such as making it possible to include ideas or display their headlines using RSS feed should be considered. Another possibility is the creation of idea-specific „campaign buttons“ that interested persons can post to

5

their webpages, preferably with some additional information such as days-to-vote or number-of-votes.

Sample: news from websites of government and president of Estonia are aggregated to the frontpage of the state portal eesti.ee using RSS:

Simplified idea process, adjustable quorum level Problem: interviewed users mentioned ideas that have passed with low number of votes as one of the reasons TOM lost its reputation, at the same time petitions on legislative issues akin to those promoted on TOM have attracted large numbers of voters and have had a positive political impact before similar ideas on TOM have even moved from the debating stage to the voting stage.

Currently one single positive vote is enough to send a TOM idea forward to the government, meaning that ideas coming through TOM are in fact not much different from a citizen’s petition. At the same time we are not able to suggest a rule that could be hardcoded into system, so the only option is to make it changeable during site operation. The

6

same goes for the overall idea process – it has a predefined duration that limits the number of possible uses for TOM. Perhaps the two could be connected: •

the possibility to get from submission to voting phase within shorter period

•

but with quorum level that makes sure ideas are really supported by users

This should result in flexible targets – for example a 30 day idea process would require the number of votes that X% of ideas during previous Y months have reached, but bringing the process down to half that time would require N times more votes etc…

Sample: petitions.pm.gov.uk lets user choose the length of petition period; other notable fields are category and possibility to choose human-readable URL so that is easy to forward by email and short enough to be published on paper.

Idea pre-selection Problem: some of the ideas entering the system have been already answered as well as clearly outside the scope of state action – and while it shows the public interest in the subject matter, passing them through the complete system is an additional burden on users and civil servants 7

•

Possibly a back-stage where ideas are prepared, only getting pushed to frontpage if they get enough user attention.

Similar solutions exist in various social software platforms – for example kuro5hin.org has an editing period before articles are pushed to the frontpage, digg.com requires number of users "digging:, i.e. promoting a news item for it to move to frontpage etc.

This requires a sizeable number of users, though, and current usage levels of TOM are clearly not sufficient for such a multi-stage solution.

Sample: kuro5hin.org's voting and editing queue, visible to logged-in users who can help the author make corrections and decide if the story is worth being promoted on the frontpage.

Rejected ideas, reasons for rejections Problem: administrators can delete profane, libelous etc ideas in the current TOM tool, but there is no feedback to the user community about deletion – what is allowed and what not; at the same time there is no solution that would allow admin (or community?) to reject an idea with clear explanation of the reason when it is duplicate, has been already answered etc

8

•

Provide a list of deleted/rejected ideas with o

the reason for rejection

o

the text of the idea visible or hidden (if unsuitable)

o

user comments about the deletion, so that they can protest the deletion

A similar solution is in use at http://petitions.pm.gov.uk/list/rejected - and the number of rejections appears to be circa 50% of submitted petitions.

Additionally, ideas rejected because they are duplicates demonstrate the public interest in the topic, which might be used by the system somehow. •

If the idea is considered to be a duplicate and thus rejected it should link to the original idea

•

Original ideas should link back to attempted duplicate ideas

In this way, the duplicates add credibility to the original idea (that might be in the commenting or voting process) and if the original idea is to be resubmitted additional information could be found from duplicates. Moreover, contacting the authors of duplicates could be one way to spark discussion or gather support for the idea.

Sample: rejected idea on petitions.pm.gov.uk with an explanation of the reason, full text not visible, also includes a suggestion to participate in another similar petition.

9

Categorization / tagging system Problem: As the number of ideas in the system is potentially large, appropriate keywords might not be present in the idea text and solutions for user activation (email when an idea is presented on one's field of interest), duplication avoidance (search for present ideas), using the system to understand public interest (popular topics, ideas related to a particular field) or in the legislative process (could any of the ideas be incorporated into a legal amendment?) need an easy and trustworthy search solution.

Tagging

ideas

with

keywords

–

both

pre-selected,

like

categories

on

http://petitions.pm.gov.uk and freeform, like used during TOM analysis to build a tagcloud – is possibly the easiest way to achieve this. For ease of use it ought to be built into TID+, for additional value, using an external solution like del.icio.us which adds visibility to ideas, makes it easy to track fields of interest using RSS feeds, find related information outside TID+ etc •

requirement to add at least a general category to the idea

•

possibility to add free-form keywords (author, administrators) 10

•

display a tagcloud on frontpage and other appropriate places

Tags can be saved to del.icio.us using their API (see http://del.icio.us/help/api/), which currently has a limit of circa 200 posts per hour. Embedding of tagcloud (tagroll) is possible using tool at http://del.icio.us/help/tagrolls Sample:

TOM

tag

cloud

visualized

using

http://kevan.org/extispicious.cgi?name=tom.al.icio.us (tags were assigned during TOM analysis, internal analysis database interfaced using del.icio.us API to save all ideas with tags)

Idea recycling Possibility: Using an old idea as the base for new idea to take it forward based on the response for the initial idea; giving the ability to contact users who participated in commenting or voting on the original idea to help edit, promote, vote on the new one.

11

•

add the possibility to start a new idea based on an existing finished idea (may be voted in or out, answered or not)

•

add the possibility to contact participants of a previous idea (after a new idea has been accepted into system)

This creates a possible scalability issue (what if one idea has 1 mil voters?) that needs to be considered, as well as a change in user terms (although currently it is permissible to contact users „to send information related to the idea“).

User authentication Problem: User authentication and public display of user-related information should be pseudonymous, as according to the TOM analysis a large proportion of incoming searchengine traffic is the result of using real names as keywords. At the same time, users might prefer to participate under their real name to establish a reputation or link their user account to external systems that they consider representative of their reputation.

•

users should be able to decide how they are represented on TID+

•

support for standards like OpenID that allow for pseudomymous authentication and relations to outside reputation providers (users' weblog, authenticated login with IDcard etc)

Sample: jyte.com is a site where people who have logged in using their OpenID credentials can vote for or against a claim.

12

Reputation and self-moderation system Possibility: The TOM analysis revealed that users wanted to discover user activity data such as the number of ideas presented and access other ideas or comments by the same user. Adding this function should also help TID+ users create an in-system reputational yardstick. Publishing key statistics about users and listing their recent activity is common in forum software, wikis and elsewhere.

•

add user’s ’profile page’ with stats and links to activity

•

add key stats next to user’s name/pseudonym in comments, idea header (if author)

The possibility of using the user’s reputation in enabling TID+ self-moderation should be considered. During the analysis phase different options were considered, but there was no evident easy solution so we cannot give more suggestions than: •

users should have the right to moderate comments (unavoidable if the system is used by a large number of people)

13

•

self0moderation should be multi-level, to catch users who use moderation to silence opposition – like the karma function on http://slashdot.org

Sample: user's profile page from Dell's ideastorm.com, a site where people can present and vote on ideas about improving Dell products and services.

Trackability Possibility: There needs to be a unified solution providing „trackability“ – meaning that users must have the option (or perhaps the right to opt out) to be informed about the progress of the ideas they are involved in. Notification should be mandatory for author, possible for commentors / voters / interested users - another possibility is that of flagging an idea or subject field for notifications, like watchlist on Wikipedia.

14

•

notify authors always

•

commenters should be invited to vote unless they opt out

•

watchlist for ideas and subjects

The best way for implementing notifications is to make them universal – users can subscribe to e-mail alerts or follow custom RSS-feed that lists all the events they are interested in.

Sample: links under micropersuasion.com posts are produced by RSS customization service feedburner.com that makes it easy to add links to digg.com, del.icio.us etc but also allows e-mailing the post and subscribing to RSS feed using e-mail.

15

Software related requirements One of the main aims of the TID+ project is to transform TOM into a software solution deployable internationally. This means the software must be licensed in a way that allows both free distribution and changes by third parties should they wish to localize it or integrate it into existing systems. At the same time competition from other free software projects must be recognized and TID+ should be made as easily deployable as possible meaning interested parties can experiment with it before making a decision.

License TID+ should be made available as open source software with a license similar to GPL (freedom to use, free access to source code, right to modify and re-distribute). The exact license should be agreed during actual development, depending on the licensing requirements of software components used.

It is important to understand, that the "original TOM" was based on a free software platform - the Amphora document and project management system built on Python and Zope - but it has been modernized numerous times without specific requirements to make the add-ons and further development available as free and open source software. TID+ project must either reach an agreement to get the latest TOM codebase released under suitable license or rewrite the entire system.

Internationalization The TID+ features accessible to end-users (idea presenters, administrators etc) must be localizable with reasonable effort; the system must be provided with at least English localization done (including documentation and built-in help for users).

TOM has been produced for the Estonian market with no consideration for possible expansion, meaning that while the base platform may have English language resources

16

available the TOM-specific parts of the user interface are either hard-coded or available only in Estonian. Language-specific information should be separated from the design and made translatable as a string table, preferably with a web interface. Current realization of TOM may require having several such tables, in which case it is important to provide documentation describing all components that may need localization and also a tool to distribute and install these localizations.

Skinnability Changing the visual appearance of the system and integrating it into different environments should be manageable with reasonable effort, namely a single generic visual solution to be delivered with system with license similar to software.

All versions of TOM have kept the content and presentation separate, so this mostly means documenting the components that constitute the visual appearance and the rules for changing it, if possible then together with a simple tool to apply the set of presentation components.

Easy installation TID+ should be delivered either as a package that can be easily installed on a typical web hosting service or 'software appliance'. the current TOM software runs as a single custom installation, while TID+ should be installable and updateable - at least for trial purposes - by moderately experienced users.

Depending on the complexity of the system various approaches can be used to achieve this target.

For example, trixbox.com offers Asterisk open source VoIP PBX with applications for configuring and extending it as 'software appliance': downloadable ISO CD image that will install on a typical PC an operating system, all required application software and

17

perform basic configuration that result in a system capable of reasonably secure operation and updating both OS and application software.

While this might be the best option for complex systems that set specific requirements for environment and need tight cooperation of different applications, hopefully TID+ will end up much tidier and can be distributed as a package with an installation script that can be used on typical web hosting service.

Scalability The system must be able to support at least 100 000 votes per idea (current petitions in Estonia outside TOM have a similar amount of participants, petitions.pm.gov.uk has over million), a solution must be foreseen to distribute the load using a solution like Squid caches.

Privacy with accountability System must allow pseudonymous usage to avoid leaking user information to search engines while using some form of user reputation system to allow self-moderation.

Search engine optimization and usability TID+ must be accessible both to humans (including those with disabilities) and machines (search engines).

A good guide to achieving both is Webcredible Handbook, available from http://www.webcredible.co.uk/user-friendly-resources/white-papers/handbook.shtml

18