UK Web Focus (Brian Kelly)

Innovation and best practices for the Web

It’s A Walled Garden

Posted by Brian Kelly on 31 Aug 2007

What would you say about a service which:

  • Replicated resources meant for sharing
  • Had very little structure to be used for resources
  • Had long, application-specific URIs
  • Required the user to change the URIs of a resource if the appearance of the resource was to be modified
  • Made repurposing of resources difficult
  • Often hides resources behind an authentication barrier
  • Uses proprietary software to host the service

I have heard the expression “walled garden” used to describe services which, although they may be popular with their users, makes it difficult for the content to be reused.

So what service and I describing here? The answer is the JISCMail service, which is based on L-Soft‘s Listserv software, which is illustrated below.

JISCMail Web interface for the website-info-mgt list

In more detail, the ways in which the service acts as a walled garden, making interoperability with other services difficult include:

  • The main entry point for list archives on the JISCMail Web site does not necessarily provide a citeable URI. For example if you go to JISCMail’s home page and search for the website-info-mgt list you are taken to the address http://www.jiscmail.ac.uk/cgi-bin/quicktype.cgi?m – which does not provide the address of the list’s archive.
  • The URI for individual messages changes if the structure of the list archive is reorganised. For example, as can be seen in the accompanying image access to archives in by year for 2006 and earlier. Unfortunately this change in the user interface resulted in links to messages before the interface was reorganised are now broken – thus resulting in loss of citation links to potentially valuable posts which may have been references in peer-reviewed publications.
  • The lack of structure provided for list archives mean that off-line browsers, which enable related areas of a Web site to be downloaded to an off-line browser cannot be used.
  • Links to individual posts break well-established guidelines which require URIs to provide resource locators which are independent of the technology used to access the resource. A typical URI for a post in a JISCMail Web archive is of the form http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0706&L=website-info-mgt&T=0&F=&S=&X=40A26C40166C54F05D&Y=b.kelly%40ukoln.ac.uk&P=155
  • Messages in JISCMail Web archives aren’t been indexed by indexing robots, resulting in potentially useful information being hidden from popular search engines such as Google.
  • The content of the JISCMail mail archives is not being archived by the Internet Archive, thus resulting in the potential loss to a service which provides a global Web archiving service.
  • Although RSS feeds are available for mailing lists, in practice their functionality is very limited, as (a) only the subject line of individual posts is provided, and not the full content or start of the content, as is normally the case, meaning that the user has to visit the Web site in order to see if the post is of interest and (b) authentication is often required in any case before the RSS feeds can be accessed.

In addition to the limitations of the mail archive provided on the JISCMail Web site, use of email itself also has several limitations:

  • Unnecessary duplication of information: e,.g. an attached file sent to a mailing list is replicated, leading to additional disk space usage and maintenance difficulties if the resource is updated.
  • Email lists are prone to spam. Although JISCMail has a good reputation in filtering out spam, it appears that increasing numbers of users are turning away from email because of these limitations.

Does this mean that JISCMail is of no use? The answer is most certainly, no. I am a member of several JISCMail lists, and have been a subscriber since the service was launched – and of its predecessor, Mailbase. And clearly JISCMail is well-loved by many of its users.

But when the term ‘walled garden’ is used to refer to new services it is important, I feel, to apply a similar level of criticism to existing services. And, as with JISCMail, this is not necessarily a clinching argument, as there are factors such as popularity with the user community which need to be recognised.

On the other hand, in response to a post on Email IS Dying the initial two responses felt that:

I agree that email is dying. Many of our students no longer check their inboxes in the same way they don’t check their pigeon holes, but MySpace and Facebook (and Bebo) combined are small potatoes in comparison with the traffic going across IM and SMS. Microsloth messenger was the “killer app” after Netscape.

and

I agree with this entirely. We’ve stopped sending out mass emails to our students because they simply don’t read them! Online noticeboards, forums and the social networking sites are much more effective. We don’t utilise IM and SMS as much as we’d like too (yet!) but this is certainly the direction we’re heading in to communicate with our students.

So perhaps the lack of interest which seems to being shown by growing numbers of students, coupled with the limitations in interoperability provided by mailing list software means that mailing lists will soon meet Gopher and Usenet in a repository of obsoleted software.

Advertisements

16 Responses to “It’s A Walled Garden”

  1. You say: “So perhaps the lack of interest which seems to being shown by growing numbers of students, coupled with the limitations in interoperability provided by mailing list software means that mailing lists will soon meet Gopher and Usenet in a repository of obsoleted software.

    Except of course, as you point out, the JISCMail archives can’t get into that repository (in the sky?) because of the closedness of the application!

    We can argue about the relative importance of mailing lists currently but there is no doubt in my mind that they form one of the most important records of our recent digital history in terms of community discussion.

    It continues to amaze me me that such a record is both hidden from Google and not stored in the Internet Archive.

    Who is making the decisions here? Lawyers?

  2. James Clay said

    Who is in charge of JISCmail?

  3. I’m a huge fan of email as a communications medium. It just works.

    Sure “solicited” mass mails don’t get read, I don’t read mass mails that come through my letter box as snail mail either, but snail mail still has its purpose.

    As for unsolicited mass mailings – do people truly believe that when facebook and the like are more than minority players it will not have the same problems.

    Sure some email software is horrible, clunky and ill thought out. But, and this is the point you post is missing completely, email is built on standards that are well implemented, well understood and provide longevity. That is why it is not a wall garden.

    If you are stuck with something like JISCMail as your mailing list server with it then use a service like gmane.org to provide a decent interface to your mailing list and don’t bother with the JISCMail web interfaces. Try doing that with any of the other “email killers” that don’t use open standards.

    The problems you list, all of them, are in the software, not in email itself. Lets look at them:

    * Replicated resources meant for sharing

    How can you share, efficiently, without replication. What about proxies? What about Google Cache? What about the Way Back Machine? EMail lists only have one official archive, there is no replication unless a client chooses to replicate it. Email is no different from any other online communicaiton medium in this respect. It is a digital resource and can be easily replicated, but there is only one original version.

    * Had very little structure to be used for resources

    EMail itself does not impose a significant amount of structure. But sensible processes for the use of email do. Web forums (as just one example) do impose more structure, but people can (and usually do) break the structure through misuse. I had to think long and hard about making this a comment or a blog post – where’s the structure in that?

    * Had long, application-specific URIs

    Clearly a problem of the software which creates the URIs you are using. Each individual message has a world unique identifier. URIs need not be application specific nor do they need to be long.

    * Required the user to change the URIs of a resource if the appearance of the resource was to be modified

    EMail is a communication medium. It captures what I said, as the author, at any given time. The last thing I want is for someone to then come along and edit my words claiming it was the same resource. Similarly, it would not be correct of me to go back in time and change my words – once edited an email is a different resource. If you want to create a resource intended to change, email is not the way to do it.

    * Made repurposing of resources difficult

    This is merely a problem of the archival software. A decent archive will present the data in standard formats that can be repurposed in consistent ways by software.

    * Often hides resources behind an authentication barrier

    Some communications are private, some are not. I want my private communications to be protected behind authentication. I want my public resources to be free to access.

    * Uses proprietary software to host the service

    And that is automatically bad?

    My point is that EMail is the most efficient tool for the purpose of communicating to large numbers of people and for ensuring that those people can get access.

    I accept that some email services a very poor (and JISCMail certainly falls down in many ways)

    I accept that misuse of email (i.e spam and an ignorance of what we used to call netiquette) causes problems in ensuring email is useful.

    What I don’t accept is that this means email is flawed. It is our use of the email tool that is flawed.

    EMail is dead? Yeah, and video killed the radio star.

  4. And just for completeness Usenet is far from dead. I’ll give you Gopher though ;-)

  5. Ross, you conflate the two issues which Brian raised:

    a) JISCMail is a walled garden + other problems
    b) “email is dying”

    You go on to address Brian’s points about JISCMail in the context of email, which to me appears mostly pointless – it just confirms Brian’s initial assertion.

    I suggest you re-read the post.

  6. Sorry, but I’m going to pick on Ross again here – he says that the nature of email is not flawed, it’s our usage – I don’t buy that. An electronic system which has the built-in ability to drown me in spam is flawed. Full-stop. To stretch an analogy, it’s like saying that guns aren’t flawed, just people’s usage of them.

  7. Ross Gardler said

    Phil,

    I agree there are technical problems with email (namely spam), but this is growing pains. Technology and the law will sort it out eventually (in fact over 95% of spam does not reach us right now).

    My position comes from one of extensive experience in large and successful communities that are built around email. By successful I mean productive. I am a very active open source software development working in communities with thousands of members across the globe.

    Many of those communities have experimented with other mediums like social networking and blogs. They have always returned to what works best for *discussion*. The best tool for discussion is email (there is some argument for forums, personally I only like push technologies, but a mail list with a forum like web interface is the goal – gmane.org, google groups, yahoo groups etc.)

    Note me emphasis on discussion. I am not claiming that social networking tools, blogs and other tools are useless, only that they are flawed as communication channels (perhaps I ought to further refine that as open communication channels since social network tools work just fine if used for a closed, invite only network). I’ve blogged about this many times, such as at http://involve.jisc.ac.uk/wpmu/oss-watch/2007/08/21/communities-cant-flourish-in-walled-gardens/

    I think I am so comfortable and productive with email because I am a techie. I set up complex mail filters and tagging rules. This means that even though I get upwards of 1000 mails per day, if you want my attention on a public list, just put my name in it and I will see it straight away (interestingly this is exactly how I know if you send me a message on a social networking tool, i.e. via email)

    Like I said it’s the process of use, not the technology that is at fault. That doesn’t mean that stopping spam via filtering is a good thing, we need a better solution (it is coming). In the meantime, if you are drowning in spam I suggest you use a decent mail server that will filter spam, if you can’t find one then try a mail client with good spam filters – my sever filters around 300 emails a day, my mail client a further 100. I tend to see maybe one or two a day – less than I get through my letterbox at home.

  8. I agree with Phil Wilson that two issues are being conflated here. Email may or may not be dying – I think we are discovering that it is what it real usefulness is and that other modes of communication can be more effective. However this has little to do with whether jiscmail is a walled garden or not. I think it is and I think it should be. It is a service that has been set up by the educational sector for its own purposes. There is no particular requirement for it to make its archive public via Google or any other method – in fact I would strongly oppose the making of the archives of a list I manage public as it deals with commercially sensitive issues, namely the operation of timetabling software. If I were to be told that this archive was to be made public I would have to remove it.

    Also, the security measures that jiscmail have put in place mean that their lists are very rarely spammed.

  9. Ross Gardler said

    I do not intentionally conflate the two issues. Allow me to separate and summarise my points:

    – JISCMail is not closed. I agree that it has a horrible set of interfaces and the JISCMail provided archives are next to useless. However, because it uses open standards I can choose to open up selected JISCMail lists if I want to, or I can simply download the complete archives and search them locally using my client. I can’t do that in any controlled way (or even at all) using any alternative means of having online discussions.

    – EMail is not dying. To get any real work done, email is where most people currently turn. Until alternatives offer a truly open standard that enables me to do what I want with my own data then email will remain the only meaningful way of communicating with disparate groups. Such a facility may come along and kill email, but it isn’t going to be the proprietary formats of Facebook et. al. Open standards mean that even poor software implementations can be be put work effectively because we can work around the limitations of the software provided.

  10. Si Biles said

    Hi,

    I’m responding on my own behalf here, not on behalf of JISCmail, although I do, currently – work with them. ( Ok. That’s my bias out in the open … )

    JISCmail has 6500 mailing lists, with approximately 512,000 users. We run the service with limited physical server resources on a tight budget. The historical reason for the lists not being indexed for Google etc. is because of the immense strain that doing so would place on the hardware. This would push us outside operational limits and risk our ability to deliver the service quality ( 99.9% ish uptime … ) that our users (a) require and (b) have come to expect. ( Oddly enough we were debating the risks and benefits of this in house only a few days ago, and we decided against for now at least … )

    By definition JISCmail is _not_ actually a _public_ resource, it is an Academic one, and as such, it does it’s job admirably. I happen to agree completely that using the web interface is like chewing wasps, but within the confines of Listserv, this is actually really pushing the limits of what is possible. There is a huge amount of bespoke code to try and mitigate what is a really poor interface. If you have a look at some of the mailing list interfaces provided by Open Source Solutions such as GNU Mailman, you will actually gain quite some appriciation for what has been achieved with it !

    Conversely, if you look at the statistics that we have ( and you don’t – unfair, I know ), less than 5% of our users actually bother to _use_ the web interface. For a majority, they manage their subscriptions, lists etc. by e-mail – which kind of makes sense when you consider that JISCmail is a _mailing list provider_. ( This also adds a little weight to the argument that e-mail isn’t really dying off … )

    I don’t personally subscribe to a huge number of mailing lists, but to say that e-mail is dead is a ridiculous notion. It seems to me that you have either never been at the dirty end of running a corporate or insitution e-mail server – I’ve done both – and recently I was on the phone with a friend when a misconfiguration of one of the servers that he was working on caused 50,000 e-mails to dissapear into the ether in the course of about 12 hours. The estimated costs of the error were placed somewhere between £100,000 and £250,000 – in lost productivity, contracts and remediation effort. E-mail is, and will continue to be – for at least the forseeable future, the lifeblood of the internet.

    Students don’t read e-mail from Universities, staff don’t read e-mail from SysAdmins, nobody reads mail from Microsoft – it is because it is usually boring and full of rubbish that isn’t relevant to the individual – it is a case of “the boy who cried wolf” – when it is important it is thrown out by the auto-delete becuase it has no percieved value. This is the fault of the organisation, not the end user. I used to have a trick if things were really, really urgent, of using titles involving “Free Beer”,”Free Food” or “Pub Visit” if I actually needed to get all the developers that I used to work with to read a message.

    We suffer from information overload, and this is something that JISCmail counters beautifully. If you want to discuss the most arcane possible details of just the subject of “underwater basket weaving” you subscribe to that list – nothing else, low spam, low viral count, other people who are interested. JISCmail doesn’t need to be all things to all people as it does what it says on the tin incredibly well.

    Hey, but for reference we do care ! Currently there are a number of projects on the go to improve the services that we give to customers :
    – we are looking at upgrading the ListServe software to give a better interface and functionality.
    – we are looking at providing enhanced services with regard to list homepages, file repositories etc.
    – we are examining online forum/discussion software
    – we are starting to experiment with interactive 3d environments ( seriously cutting edge stuff here ! ) to integrate existing services with the Education UK community in SecondLife.
    – we have allready implemented Shibboleth Single Sign on security.
    – we are experiementing with “mashups” of Google Maps and our userdata.

    Maybe mailing lists aren’t sexy anymore – not the latest fad – but one way or another they have been around through more fads than most things, they still do exactly what they set out to do, and it will continue to do so as far as I can see into the future.

  11. Hi Stu
    Many thanks for your comments.
    You say “the web interface is like chewing wasps” (which reflects Ross’s comments that “it has a horrible set of interfaces and the JISCMail provided archives are next to useless”. You also go on to say that “JISCmail is _not_ actually a _public_ resource”.
    I would agree with you. And, as I said in my original post “Does this mean that JISCMail is of no use? The answer is most certainly, no. I am a member of several JISCMail lists, and have been a subscriber since the service was launched – and of its predecessor, Mailbase. And clearly JISCMail is well-loved by many of its users.”
    I would say that we are in agreement that JISCMail provides a valuable service to the JISC community of email users. However problems occur when it is used in a wider context e.g. by the standards development community who need to cite stable URIs. This was the jist of my comment about it being a walled garden – as Simon Geller said, that is a valuable feature of the service. I’m not using the term ‘walled garden’ in a pejorative service – instead it’s intended to flag the strengths and weakness of use of the service in a variety of contexts (and how its strengths in supporting niche communities interested in ‘underwater basket weaving’ need to be complemented by other services which allow contacts to be made outside the JISC sector.
    And I’m pleased to hear that the user interface limitations which you and Ross mentioned are being addressed. I hope that the development extend to addressing the URI problem.
    Thanks again

    Brian

  12. Si Biles said

    Brian,

    Thanks for the response. ( It’s Si by the way … )

    I do think that the interface is unpleasant, but having said that, I go on to point out that it is probably the best implementation of a mailing list manager that I’ve seen, either commercially or open source. And to be honest, if it wasn’t for the fact that I have occasionally to do things with it that trancend my usual mailing list tasks, I wouldn’t have a problem with using it, basic functionality is readily available, it is the more complex less frequent things that remind me of insect eating …

    The trouble is, that irregardless of software, programming an interface to manage such a large number of groups with such a large number of users doesn’t really lend itself particularly to a web page. If you have suggestions as to how to improve it, we have always been open to constructive criticism – infact we ask for it !

    The URI problem is an issue, and one that we are well aware of – it is a pain in the neck and I can see that, for people wishing to reference a single post in another resource, this would be a real sticking point. We’ll see what we can do.

    Thanks,

    Si

  13. Hi Si
    Sorry for getting your name wrong – I must increase ther font size in my browser (or change the prescription for my glasses!)
    Thanks for yopur invitation for feedback. For info I was on JISCMail’s Advisory Group when it was set up – so I’m well aware of the challenges you face in managing such a large service. And I know the quality of the email service is excellent (spam-free lists :-) as is the user support.
    The problems, IMHO, are with the Web interface. This may be a minority area of interest, as you suggest. OTOH the low numbers of users of the Web interface may be due to the user interface problems you’ve mentioned. However my main areas of concern (which results in the closed garden nature of the Web side of things) relate to the Web interfaces. Primarily these include (a) the application-specific nature of the URIs (b) the lack of structure of the URIs (c) the ability of list owners to change URIs (d) difficulties of citing URIs of messages (e) poor implementation for RSS and (f) the more general difficult of interoperating with third party services.

    I know the URI issue is a feature of L-Soft’s dynamic access to the email store. This has the advantage of providing up-to-date access to recent messages. However there is a significant cost to this. I always liked Mailbase’s interface to its Web archives, with URIs of the form http://www.mailbase.ac.uk/lists/web-support/message-001.html
    This was understandable (by humans), easily remembered, was independent of the technology, etc. Would it not be possible for JISCMail to craete a simimilar statuic file store for its archives, to complement the dynamic interface. This would provide losts of benefits to the users, and would fit in more closely with JISC’s Information Environment architecture.
    If this was possible, this would address my concerns (a)-(e).
    I don’t understand why L-Soft’s sofwtare only syndicates the subjet messages and sender and not the first few sentences. If you can’t persuade L-Soft to fix this problem, you would be able to produce an RSS feeds of the static archives relatively easily (in terms of the architectutre – I don’t know about the scalability factors).
    Note that related to getting messages being found by Google, and other robots (e.g. prteservation agents) I know about the performances factors. However Ross MacIntyre (MIMAS) spoke to me last week about the approaches MIMAS are taking to exposing their dynamic data to Google. So you might find it useful to talk to him.
    The final issue is realted to the fact that not all of your users will want their data made open. WOuld it not be possible for listowners to set a switch, which would allow their archives, RSS feeds to be made freely available, thus providing user choice.
    The final suggestion I would make would be to ensure the users are aware of the limitations of the existing system. I suspect many users aren’ty aware that changing the date structure of mail archives changes the URIs – so any message cited in scholarly publications (“here’s the URI of the message announcing the invetion of cold fusion”) will break. This is a major problem for a service aimed at the academic community, so if the problem can’t be fixed, let’s at least ensure that list owenbers and JISCMail users geenrally, are aware of the dangers of doing this.

    Hope these comments are helpful

    Thanks

    Brian

  14. Ross Gardler said

    Thanks everyone for your comments, all very useful.

    I just want to point out that if people have a JISC Mail list that they *want* to make available to people outside of the normal JISC communities this can be done with minimal impact on the JISCMail servers (assuming my proposal is not in contravention of the JISCMail terms and conditions).

    Simply subscribe your list to a service like Google Groups or Gmane.org. The extra impact on JISCMail servers is just one more subscription.

    Personally, I recommend gmane.org as this allows you to upload arcives as well as capture future postings.

  15. Si Biles said

    Ross,

    Please don’t treat this as a definitive answer but …

    Section 7 of the Acceptable Use Policy states :

    “Archives or collections of the messages sent to a JISCmail list may not be made publicly available at another site unless the listowner has granted explicit permission, and the list members have been informed.”

    Bearing also in mind that there are copyright issues at stake …

    JISCmail copyright guidelines also state :

    “Respect copyright when forwarding messages, if in doubt check with the author.”

    Explicit permission for JISCmail to handle lists is given by the AUP :

    “You give JISCmail permission to reproduce, deliver, distribute and archive your message as appropriate, and you agree that other JISCmail users may read, save, download or print your message.”

    But I don’t know what permissions would be required of a copyright holder in order to reproduce elsewhere …

    I agree that it is an elegant technical solution, however, as it is potentially possible to subscribe such a “public” user to a “private” list, I would think that overall it might be best avoided without careful legal advice …

    Si

  16. laurel hill Proxy solicitation job

    It

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

 
%d bloggers like this: