A social perspective of custom content and user tracking via magic cookies

The final goal of user interface design is to make the process as transparent as possible. These transparent expert decision systems save time and money for consumers and marketers. However, if a system is allowed to analyze and understand user habits using expert systems, the user may gain convenience at the expense of their privacy.

The two sided nature of a web interface increases the number of design factors need to be taken into consideration. In addition to the goals and preferences of the user, the owners and operators of a particular web site will have other goals impinging on them. These goals might primarily include attracting and maintaining users within their system, or perhaps to lure the user to advertisements which provide the revenue to maintain the rest of their site.

The web presents a huge improvement in the ability of the designers to perform usability studies. The author of the interface can use many methods to track the activities of all the users who ever use their interface, not just a select group or test bed (Greenspun).
However, because of the design of Internet protocols, one user may appear to come from many logical (or even physical) locations (such as a dynamically dial-up Internet address), and many users may all appear to be coming from a single logical location (a single company firewall or proxy server). However a technology known as "magic cookies" in web browsers are one way for a server to store state information on a particular instance of a client ("Persistent"), and keep a closer eye on individual patterns and habits.

A cookie is given to the server with a Set-Cookie string in the header as part of an HTTP response ("Persistent"). Cookie functionality has to be programmed into the web server to server static pages, so typically a CGI script will be used to generate the pages and the headers ("Persistent"). This allows for easy creation and retrieval of cookie state information. When a user returns to a web site, the server can check this cookie, see what state information has been left since the last visit and easily change it if necessary (Mulligan).

Even when it's known that the user's browser can accept cookies, the designer must still be careful about how they are used. The Netscape specification says that a specific site is only entitled to 20 cookies on the user's machine, each of which are limited to 4,096 bytes ("Persistent"). Since these cookies would only be likely to contain tracking identification, instead of the actual information of a user's profile, the size limitation is not usually a big issue.

It should be noted that the browser client is only required to store 300 cookies in total, and when that limit is exceeded may start to delete the cookies in order of least used ("Persistent"). Also, all cookies are set to expire at some specific date and time. This date may be set decades ahead, but one day it will expire (Smith). If the site using cookies has a number of transient users who drop in every few years or visit a number of other cookie-intensive sites between visits, any information the server administrator can cull from the cookie logs will not be as accurate.

A big concern, however, is that not all browsers support the cookies (Greenspun). Lately this has become less of a concern as the two leading browsers, Netscape Navigator and Microsoft Internet Explorer do (Barr). Cookies have also been accused of becoming "useless" once they were made optional to accept because it's so much more difficult to design a site with cookie-enabled and cookie-less browers in mind at once (Garaffa). The designer must be aware of this and design a site that is personalized for users who do use cookies and returns to it's generic form when nothing is returned.

A report by the Center for Democracy and Technology (CDT) noted that so many users on the web are using cookies for purposes such as storing passwords and making purchases on e-commerce sites that disabling the function of cookies entirely "is not an attractive option" for them ("Statement"). However, fifty-two percent of respondents in a Bell Lab survey said that they were concerned in some way about cookies (Cranor). Which reports the designer lends credit to should determine how much design effort and requirement is put into the use and presence of cookies.

In the web site Slashdot, however, many users turn on their cookies for the sole reason of participating on that site. Even on this site for privacy conscious and technical savvy geeks, the owner of the site refused to back down on his policy of using them (Malda). This is likely because they are quite good at maintaining user state information when used correctly. As in other cases, cookies simplify the user interface by making the exchange of user authentication information transparent. If the browser is constantly sending the users information to the Slashdot site, then a number of scripts can be referenced using only the name of the program in the URL (Malda).

However, in the majority of cases an individual site will not be enough to encourage a privacy conscious user to re-enable cookies.
The Bell Labs survey stated that almost 56% of the respondents who were aware of cookies had changed their cookie settings to "something other than accepting all cookies without warning" (Cranor). This might mean they could deny or selectively accept the data that the server is sending them to facilitate moment. Sites which require custom content to function should choose another method of authentication such GET strings contained in URLs.

Custom content is the goal of the server owner, and also satisfies one of the requirements of good system design by making the user feel right at home. Cookies allow the server to greet the user with a personalized message with their name and provide any new information that might be of interest to them on that site (Barr). A web site can send a cookie to a user's browser and give it a unique ID number that corresponds to you (Rodger, "Activists"). It can then collect data on habits and patterns, and have them ready when the user appears again and their browser returns that particular cookie. User tracking with cookies and learning systems can allow users to get the info they need without actually having to spell out their preferences (Rodger, "Cookies").

There are two distinct ways to use a set of cookies, although both can be used concurrently. In a simple case, cookies can be used during the course of a single "session", or purposeful visit to a web site, to hold a list of items selected or similar data until the user leaves and shuts down their browser (Festa). Also, the server can put what's called a "persistent cookie" on the client side, which would be in the form of a User ID number or similar information to preserve a user name and password in between visits so that the user can quickly logon and get to what they need (Festa).

Information that is collected about the user can point them to parts of the site that they might be interested in or perhaps new things that they haven't visited yet (Greenspun). The concept is not too different from the time-tested method of direct marketing, where the advertisers focus particular products on users who have shown a potential interest by buying something similar previously (Bloomberg).

Once data is collected and aggregated, it can be used to tell users what sort of books they might like, or tell them what other people in their organization or domain name also purchased. User concerns on this particular issue should be relatively low as data aggregation and presentation isn't "illegal or even inherently unethical" at this point (Seglin). However, it's important to note that individuals have a reasonable expectation of privacy in that the personal collecting the information will only be using it for the stated purpose and not sharing it with other agencies (Mulligan).

An option was set up in Netscape to prevent against the images from other Internet domains planting cookies on your machine, but the major advertisers and the companies they were working with got around this in a manner of weeks. They now use what are known as "forwarding domains" so that the domain appears to be coming from the primary site such as their top-level domain with a sub-domain that is actually the advertisers server computer, instead of a third party server ("Netscape's"). These tricks rendered the new option much less useful to users.

Failure to respect user's privacy might result in user distrust and a fear of a particular interface or system. In fact, several news articles have had opponents of the Doubleclick user profiling process call it "illegal" even though the actual legality of the process is uncertain ("Doubleclick's Big"). One of the more pressing concerns with this is brought up in the Mulligan report on privacy when she writes:

The use of cookies to correlate personal date gives advertisers a much higher chance of getting a response to ads they present, and therefore higher revenue (Tedeschi). Soe people might be uncomfortable with the realization that they are being targeted, but since they will be seeing the ads anyhow, targeted ads are more valuable to the user and the advertiser (Festa).

This is no small consideration in the design of the user interface. Click-through rates on banner ads are declining and causing potential advertisers to re-think how much they are willing to pay sites and content providers for space within their pages (Tedeschi). Advertising companies state repeatedly that targeted advertising is necessary if the current paradigm of the Internet interface is to stay free and valuable (Rodger, "Privacy"). The revenues to allow the sites to continue operating and serving the user are just as important as any other factor in getting the user their information.

One study indicates that Internet users are also worried that they'll be tracked through the use of persistent identifiers like cookies (Cranor), but others like Robert Smith of the Privacy Journal feel that their base expectation is one of privacy, particularly that their browsing habits are highly unlikely to be combined with their real names or physical locations (Rodger, "Activists").

User perception over the actual function of cookies was varied and inconsistent in the Bell Labs study (Cranor). While it's a good indication for an interface if the user has no need to understand it in order to use it, the recent media activity might cause some users to feel uncomfortable about having cookies enabled during their browsing. If the industry isn't careful about how they do this, they could lose a very lucrative method of catering to their user base, and severely cripple a very valuable UI design for the future.

Right now there are significant obstacles to automatically collecting data and correlating profiles on Internet users, but as computer technology advances those obstacles are rapidly diminishing (Kahney). It's important for a content provider not to lose sight of how easily politics and public opinion play into advertisers decisions. When the controversy surrounding Doubleclick unfolded, large sites such as AltaVista and Kozmo.com moved their business away because of concerns over privacy ("Doubleclick's Big"). The value of consumer profiling will only increase if companies like Doubleclick don't muddy the waters with legislation first. The lack of outside consumer pressure on the companies is causing many advertising companies to ignore privacy issues at this time to focus on revenue building (Kahney).

The Internet advertising industry is not atypical in their mindset. Spokespeople for the computer industry typically take an optimistic view of social consequences of technology and tend to couch the negative aspects in more positive light when confronted, instead of looking more closely at the actual problem (Markoff).
Industry representatives such as the senior Vice-president of e-commerce at Land's End, Inc believe that a privacy backlash is in the works (Green). It may actually be closer than they imagine. The Federal Trade Commission stated that they will take the authority to oversee e-commerce and data collection if the Internet companies don't show good faith in respecting the privacy of their customers (Bloomberg). Many local and state governments are also beginning to take action with proposed legislation and departmental press releases.

In one report, the state Attorney General of South Carolina, Charlie Condon wants legislation so that "people do not shed their privacy when they go online" ("Doubleclick Probe"). A New York office press release even before this incident was equally adamant, stating that new technologies have "placed all of us in an electronic fishbowl", and calling for new privacy legislation to come out of their office ("New Privacy"). Condon even threatened to draft proposed legislation against companies secretly compiling online consumer profiles ("Doubleclick Probe"). Even a woman in California joined in the fray and filed suit against them for for "unlawfully obtaining and selling consumers' private personal information" ("Privacy").

At this time, many European counties already have guidelines on what can be done with information once it's collected, but the United States doesn't (Clausing). Instead there is a reliance on the market to regulate the businesses to do the proper thing (Clausing). This has been a point of contention between the U.S. and other countries recently. However, the result of legislation could be very negative to user perceptions, and put intelligent learning user interfaces on the Internet at a severe disadvantage. Some consumer and market groups worry that if privacy solutions aren't allowed to evolve on their own, they will be heavy-handed and unsuitable for the nature of the Internet (McCullagh).

They believe that if the user doesn't trust the interface they are provided, through conscientious following of the posted privacy policies or similar measures, they may not return. This sort of "organic" technology and response to keep information private are already popping up throughout Internet communities. The online bookseller Amazon.com went over the limits of their own privacy policy in targeting specific users and fell under a number of class action suits in response (Green). As a result of these policy violations, many members of the large web portal Slashdot have agreed not to visit the online bookseller again until they demonstrate a less cavalier attitude ("DoubleClick DoubleCross").

The interface designer is also trying to make the end user do what they want, which is go to their advertising and source of revenue. The goal of targeting advertisements towards specific users is to predict "what you will and won't react to" (Rodger, "Cookies"), intentionally trying to influence your actions in these cases. In this way, the user interface is designed with the constraints of both the operator and the interface owner in mind, with different goals merged in the same interface.
The primary problem is not knowing exactly how web surfers feel about their travels: whether they prefer total anonymity, don't really care about their shopping habits, or just want to know when they're being tracked.

Even before anything was known about Doubleclick putting user's profiles together with their ID number, there was a hint of concern in government releases such as what came out of the Computer Incident Advisory Capability group. In a released document stating that it was not a risk for government agencies to enable cookies for more effective browsing, CIAC also noted a perception of anonymous browsing among the general user population, and the risks of another party having access to a list of sites they visit (Festa). Other studies point to the fact that users are typically unaware of monitoring when it's actually taking place (Bloomberg).

Privacy advocates are not so keen on the use of cookies. However the director of the Electronic Frontier Foundation Stanton McCandlish noted that there's nothing in the design of cookies that's harmful to the user, but like any technology it still has the possibility of being abused (Festa). Users may be cautious about cookies, because they are being used to track people's browsing habits between sites by advertising companies such as DoubleClick (Festa). When articles about cookie abuse hit the media, a lot of users are likely to disable their cookies and install filters to protect their privacy. The final decision to keep or delete a cookie file at the end of a session rests in the hands of the user (Barr).

The CDT believes that consumers will believe that cookies are benign and can be trusted in most cases ("Statement").
The function of user tracking may only be to create transparent and less annoying targeted advertising, but at the same time a large base of knowledge of our preferences and private activities is created by the companies with potential for abuse ("What went wrong"). It should be noted that 78% of the respondents in the Bell Labs survey said they would "definitely or probably agree to Web sites using persistent identifiers (possibly implemented using cookies) to provide a customized service" (Cranor). This should be kept in mind when designing any personalized user interface.
If the cookies facilitate the users' experience, they won't even notice or fear them.

Cookies are an amazingly powerful interface method and have a lot of potential for making content providers and the Internet in general a more valuable resource. However the technology is a double-edged sword, and the various data industries springing up will have to learn to respect it or government regulation, which could stifle open and innovative development of transparent interfaces. If users, advertisers and government regulators can come to agreement on proper uses and technologies, further development and application of learning interfaces on the web will proceed quickly.

Green, Heather. "1984 in 2000: Getting Too Personal: Sites are gathering fat files on your shopping habits. They may live
to regret it." Business Week. http://www.businessweek.com/2000/0006/b3667023.htm.

"Netscape’s Communicator third party cookie option foiled." Cookie Central. 1998. http://www.cookiecentral.com/dsc3.htm
(25 Jan 1998 23:48:29 GMT).

Mulligan, Deirdre. Public Workshop on Online Profiling: Testimony of the Center for Democracy and Technology Before the Federal Trade Commission. November 30, 1999. Center for Democracy and Technology. http://www.cdt.org/testimony/ftc/mulliganFTC.11.30.99.shtml.

"New Privacy Protections Sought for Information Age: Spitzer Agenda Would Expand Individual’s
Control Over Personal Data" Office of the New York State Attorney General Elliot Spitzer.
http://www.oag.state.ny.us/press/2000/jan/jan25a00.html.

Rodger, Will. "Activists charge DoubleClick double cross: Web users have lost privacy with the drop of a cookie, they say." USATODAY.com, 2000. 02/21/00- Updated 05:23 PM ET. http://www.usatoday.com/life/cyber/tech/cth211.htm

Rodger, Will. Privacy bill would control 'cookies': Complaint filed with FTC over DoubleClick's tracking technology. USATODAY.com. http://www.usatoday.com/life/cyber/tech/cth319.htm. (02/21/00- Updated 05:25 PM ET)

Seglin, Jeffrey L. Dot.Con: As competition heats up, ethics, morals, and manners get lost in cyberspace. 21 Feb 2000. Forbes Magazine. http://www.forbes.com/asap/00/0221/135.htm.

"What went wrong? So If cookies are so much of a nuisance why was they developed in the first place?" Cookie Central. 1998. http://www.cookiecentral.com/cookie5.htm (January 15, 2000 19:22:32 GMT).

Please let me know how you got here, if this page was useful to you, and your opinions.

Unless noted, all content on epistolary.org is © Copyright 1999-2008 to Rob Carlson with all rights reserved. All information is verified when possible, cited as appropriate and applied in the real world at your own risk. Send all feedback to rob@vees.net.

A social perspective of custom content and user tracking via magic cookies

Works Cited

Leave a Reply