About this Project

Three men can be seen in the foreground, two shovelling large lumps of coal into a basket.  A third man is holding a blackened sack full of coal.  In the background a barge can be seen and the figure of a man, blackened by coal dust, carrying an empty sack. A detail from W.H. Pyne, Coal Heavers, 1806. © Bristol Public Libraries.

Introduction

London Lives makes available, in a fully digitised and searchable form, a wide range of primary sources about eighteenth-century London, with a particular focus on plebeian Londoners. This resource includes over 240,000 manuscript and printed pages from eight London archives and is supplemented by fifteen datasets created by other projects. It provides access to historical records containing over 3.35 million name instances. Facilities are provided to allow users to link together records relating to the same individual, and to compile biographies of the best documented individuals.

Primary funding was provided by the Economic and Social Research Council. Project Directors are Professors Tim Hitchcock and Robert Shoemaker and the Project Manager is Dr Sharon Howard. Implemented by the Humanities Research Institute at the University of Sheffield and the Higher Education Digitisation Service at the University of Hertfordshire, this website is published by HRI Online Publications.

For guidance in using this site see the individual search pages and the Research Guides.

Back to Top

What's New (Version 1.1, April 2012)

The following changes were implemented during the first annual update of London Lives:

  • Keyword search is now the default form of search. We have made this change in order to emphasise the wide range of searchable text available on this website. All other forms of search, however, including Person Name search, remain available. Unfortunately, for technical reasons it is still not possible to combine person name and keyword searching.
  • Every document display page now includes, in the grey summary box near the top of the page, a link to the relevant background page which provides information about that document type.
  • We have provided instructions on the London Lives Wiki Homepage for what to do if you are unable to login using the same username and password you use for the main website.
  • In order to give credit to the full team that produced London Lives, our Citation Guide now includes a recommended form for citing the whole project. In addition, the list of project staff and their responsibilities has been revised.
  • We have fixed a bug which rendered some documents inaccessible.
  • Several corrections to the tagging and transcriptions have been made. We are grateful to our registered users for reporting many of these errors.

Searches produced using the site may therefore produce slightly different results than they have previously. For this reason it is important always to cite the current version number of the site (1.1), and date accessed, when composing citations to this website.

Back to Top

Project Staff

  • The directors of this project, and authors, with Sharon Howard, of all the historical background pages, are Professor Tim Hitchcock (University of Hertfordshire) and Professor Robert Shoemaker (University of Sheffield).
  • The Project Manager is Dr Sharon Howard.
  • The technical officer responsible for implementing the search engine is Jamie McLaughlin.
  • The technical officers responsible for the automated markup were Ed MacKenzie and Katherine Rogers.
  • The Senior Data Developer, in charge of all the tagging procedures, was Dr Philippa Hardman.
  • The other Data Developers were Anna Bayman, Eilidh Garrett, Carol Lewis-Roylance, Susan Parkinson, Anna Simmons, Gwen Smithson, Nicola Wilcox, and Catherine Wright.
  • The London researcher was Mary Clayton.

Back to Top

Rationale for the Project and Choice of Sources

Historians studying the lives of the non-elite have a wealth of sources available, but these tend to be produced by institutions, and analysed in that context, thereby prioritising institutional considerations over those of their users. This project makes it possible to switch the research focus to individuals by allowing researchers to quickly locate all the relevant documents for particular people from a range of sources. In doing so, it seeks to demonstrate that the pressures created by the users of government services and charitable organisations shaped the course of the development of these institutions. By examining how individual Londoners engaged with and manipulated these agencies for their own ends, this project is designed to assess the role of plebeians in the evolution of social practices in the modern metropolis.

The choice of sources included in London Lives was largely determined by this research agenda, but also by practical considerations including which sources were available for digitisation and which were most likely to contain information on the same individuals. By including externally created datasets, the remit of the project was broadened to include a wider range of sources than would otherwise have been possible, and made it possible for the names in those datasets to be linked up with other sources.

The lives of plebeian Londoners most often intersected with institutional records when they were caught up in the criminal justice system, or sought poor relief or medical treatment. Our choice of sources was designed to capture this pattern of interaction, but we also sought to include comprehensive archival collections. Where records were simply too voluminous to include, as in the case of parish rate books, we have excluded them, but wherever possible the entire archive of each institution has been digitised. Where it was not possible to include the records of all relevant archives, as with the large number of parishes and guilds, we have chosen representative examples of complete archives with good record survival.

For criminal justice, we started with the already digitised Old Bailey Proceedings, the largest printed source detailing the lives of non-elite people ever produced. We supplemented this with the most descriptive related records about serious crime available, including all surviving examples of the:

  • Ordinary's Accounts (OA): biographies of executed criminals written by the chaplain of Newgate Prison.
  • Sessions Papers (PS): manuscript documents which provide additional evidence about the crimes tried at the Old Bailey and other courts, as well as documents concerning poor relief.
  • Criminal Registers (CR): lists of prisoners held in Newgate Prison.
  • Coroners's Inquests (IC): documents relating to deaths thought to be suspicious, but which did not result in a formal prosecution.

For the most part, these records pertain to the prosecution of felonies. For petty crimes, this website has fewer records, because the surviving records tend to be less descriptive. However, the Sessions Papers include some petty crimes as well as felonies. In addition, we have included the records of Bridewell, the house of correction for the City of London, where the poor accused of petty crimes were punished.

Official responsibility for poor relief lay with London's parishes, of which there were more than one hundred, many of which have left very rich archives. The records of three parishes have been comprehensively digitised for this project:

These parishes were chosen for the quality of their records, and the extent to which they exemplify different parts of London. Each of the selected parishes had a distinctive social and occupational composition. We were able to supplement the records of these three parishes with externally created datasets of settlement and workhouse records from two further parishes:

Charity for the poor also came from the guilds and associational charities. We have included the records of one London guild, the Carpenters' Company. This was one of the less prestigious companies and included a number of plebeian members. It also distributed considerable charitable funds to its members. We have also included partial transcriptions of the registers of the Marine Society, a charity which provided training at sea for poor boys.

Medical care for the poor was provided in parochial workhouses, reflected in the parish records, and hospitals. We have included the records of one of the royal hospitals, St Thomas's Hospital, including its detailed admissions and discharge registers.

Beyond this, we have included a number of externally created datasets which, although their focus is not always on plebeian Londoners, do include information on many of the Londoners who also appear in the records listed above. These include tax and voting records, wills, fire insurance registers, and urban directories.

Background information about all the document types and datasets included in London Lives is available on separate pages. Other pages provide extensive background information about the institutions which created these records.

Back to Top

Funding

This project was made possible by a generous research grant from the Economic and Social Research Council. We are also grateful for assistance from the University of Hertfordshire and the University of Sheffield.

Awards

BSECS

In January 2011 this website won the British Society for Eighteenth-Century Studies Prize for Digital Resources, and co-directors Tim Hitchcock and Robert Shoemaker were awarded the Longman-History Today Trustees Award for their "major contribution to history over the past year or years" with the Old Bailey and London Lives projects.

Technical Methods

This section explains the methods used for the digitisation and markup of all the manuscript sources digitised for this project. For the methods used for the digitisation and markup of the Old Bailey Proceedings and Ordinary's Accounts, see the description on the Old Bailey Online. For methods used in the development of the externally created datasets included on this website, see the relevant background pages relating to these resources.

A man in a grey apron and sheepskin gaiters is walking towards the viewer, with a large bucket full of water in each hand.  In the background a row of carriages and horses can be seen. W.H. Pyne, Waterman, 1806. © Bristol Public Libraries.

Digitisation of Images

Starting with microfilms of the original manuscripts, page images were scanned to create high definition, 400dpi JPEG files. Lower-quality (smaller size) JPEG versions of these files have subsequently been created for transmission over the internet. The higher quality files have been preserved for archival purposes, and should eventually be accessible over the web once data transmission speeds improve. The one exception to this was the image files for documents held at the Bridewell and Bethlem Archives, which were directly sourced from their online catalogue.

Digitisation of the images was performed by the Higher Education Digitisation Service at the University of Hertfordshire.

Back to Top

Text Rekeying

The manuscript texts were manually typed by the process known as double rekeying whereby the text is transcribed twice, by two different typists, and then the two transcriptions are compared by computer. Differences are identified and then resolved manually. This process was carried out under the supervision of the Higher Education Digitisation Service.

With a perfectly clear and well-formed original text, this methodology results in an accuracy rate well over 99%. However, difficulties in deciphering seventeenth and eighteenth-century handwriting, fading, tears, and blots in the original, and distortions introduced by the processes of microfilming and image digitisation can reduce the level of accuracy. Where a perfectly accurate reading of the text is required, users are strongly advised to consult the original document image provided on the right of the page, or if necessary the original document in the relevant archive. The image provided here can be enlarged by scrolling your mouse over the image. The level of magnification can then be adjusted by holding down the left button on your mouse and moving the mouse wheel. Click on the full size button to access the largest available page image, which in turn can be further magnified by clicking the image.

Documents which take the form of tables are particularly difficult to reproduce in transcribed form and users will need to consult the original page images in order to understand the meaning of the text.

Back to Top

Markup of Text

The digitised text can be searched for any character string from the keyword search page, but in order to facilitate structured searching the text was also marked up (or tagged) in XML. The particular focus of the project is on names, but we have also marked up occupations, places, and dates. Names are marked up only where there is both a surname and a forename. The markup, however, is neither comprehensive nor entirely accurate and searches on this tagged information should be supplemented by keyword searches.

The markup was done by a combination of automated and manual processes.

The automated markup used a combination of dictionaries derived from the Old Bailey Proceedings, a sample of manually tagged data from this project, and natural language processing to identify names, occupations, places and dates. Genders were added to names using a dictionary of forenames. A significant number of names could not be allocated a gender with this method and are labelled "unknowns".

Manual checking was then carried out by a team of data developers (see project staff), who eliminated false positives and added some markup missed by the automated process. Owing to the time-consuming nature of this process and the limited funding available, this manual markup procedure was not applied to all documents to the same level. The greatest effort was put into ensuring that names were correctly tagged. The instructions given to the data developers are documented in Tagging Levels. Where accurate structured searching is required, users are advised to consult the tagging level description for the particular documents they are interested in, in order to determine the extent of markup achieved.

To ensure accurate searches, it is always best to supplement structured searching with a general keyword search for the term or terms you are interested in. This is particularly true of places and occupations, which were not marked up to the same level as names.

Back to Top

Search Engine

All of the search and statistics features are implemented using MySQL. The marked up texts were first processed using Saxon to create tab delimited data files. These files were then imported into MySQL, indexed and modified in order to enable the various search features. This process was carried out by HRI Digital at the Humanities Research Institute. In addition a modified version of Double Metaphone has been used to create a flexible form of search available through the Person Search Page. In the case of Surname searches, the initial letter of each name has been excluded from the workings of the phonetic algorithm that underpin sounds like searches in double mataphone. The final two letters of surnames where either of these letters is an s have also been excluded from the workings of the phonetic algorithm. In the case of Forenames, a look-up table of common forename variants has been created, and used to widen the search beyond names that sound like the search term.

Website Design

The HRI designed and implemented the interactive functionality of this website. Mark Hadley designed its visual look and feel.

Research Users

Basic use of the search facilities of London Lives is available to everyone without registration. If you wish to acquire the ability to save the results of a search in a Workspace, create Sets (groups of name instances pertaining to a single individual), assign sets to Groups, and contribute to the London Lives wiki, you will need to register to use the site. Registration is available to anyone who agrees to the London Lives Terms and Conditions.

More advanced functions are available for Research Users, scholars conducting research using the site. These include the ability to use of the automated matching facility, and, in consultation with the project, to assign roles.

Automated Matching allows users to combine all the names tagged in one group of documents (with or without a role) with all the names tagged in another group of documents within a specified time period. This function facilitates the comparison of individuals who have one characteristic, such as those found in criminal records, with individuals with a second characteristic, for example those recorded in poor relief records, allowing the relationship between these two characteristics to be explored.

Roles can be used as search criteria on the Person Name Search and Set Search pages, and identify a particular characteristic of an individual in a document, for example defendant in the Old Bailey Proceedings, or pauper in parish relief records. Only one role can be assigned to any single name instance, so this facility has to be used with some care and we ask Research Users to notify the project of any new roles they wish to create before doing so. Once roles are assigned, one can learn more about the labelled individuals by searching for other information about them by using the role as a search delimiter.

Because roles become part of the searchable material on the website (and therefore must be assigned accurately), and automated matching makes heavy demands on computing time, both functions are only available to a small number of advanced researchers. If you wish to become a Research User with access to these facilities, please email us at londonlives@shef.ac.uk with the following information:

  • your name
  • your institutional affiliation, if any, and role within the institution
  • a short description of the research project you are working on (no more than 50-100 words)
  • a brief explanation of how you propose to use London Lives in your research

Back to Top

A man sits on a red carpet with bells on his hat and shoes, and with three further bells in his hands.  A man in a white hat and reda nd white striped suit stands behind him.  A couple have stopped to watch the performance, and a fair can be seen in the background. W.H. Pyne, The Bellman, 1806. © Bristol Public Libraries.

London Lives and Federated Searching: Connected Histories

The documents in London Lives can also be searched in two complementary websites.

Connected Histories provides integrated searching of a wide range of digital resources relating to early modern and nineteenth-century British history. More than simply a portal for accessing these historical resources, this project used natural language processing techniques (also used in London Lives) in order to remotely tag previously unstructured texts and allow consistent structured searching of names, places and dates. Registered users are able to save search results and make connections between documents in separate resources.

In addition to the sources contained in London Lives, the resource includes the Old Bailey Proceedings Online, British History Online, the Burney Newspaper Collection, the Parliamentary Papers, British Museum Images, and more.

Mapping London Lives: Locating London's Past

Locating London's Past allows place name data from selected sources in London Lives to be be mapped onto a fully rasterised and georeferenced edition of John Rocque's 1746 map of London and the first accurate modern Ordinance Survey Map (1869-80). Users can compare this evidence with evidence of parish population densities; crime locations and defendant residences from the Old Bailey Proceedings; plague and taxation records from the Centre for Metropolitan History; and archaeological records of clay pipes and accessioned glass from the Museum of London Archaeology Service.

Back to Top

Advertising Policy

London Lives is a not-for-profit project whose sole objective is to make the documents and data it contains available for private use to all internet users free of charge. Since it costs money to maintain the site, and the grant which funded the creation of the website ends in August 2010, it is necessary to obtain separate funding to ensure its continuation. For this reason, the site includes advertising. All profits derived from this advertising will be devoted to maintaining and upgrading the site.

Back to Top

Acknowledgements

We would like to thank the following institutions and individuals for their generous help with the project:

  • Our major funder, the Economic and Social Research Council, and the Universities of Hertfordshire and Sheffield for making this project possible.
  • The libraries and archives (listed under Copyright Information) who permitted us to reproduce and transcribe page images of their original documents.
  • The creators of databases (listed on the relevant dataset background pages) who permitted us to incorporate their data into this resource.
  • Ed Duncan and Viki Philpott, who contributed to the project as postgraduate interns researching and writing several biographies.
  • Students taking HST 2024 Poor Man, Sick Man, Beggar Man, Thief at the University of Sheffield in the autumn semester 2009 who contributed some of the biographies.
  • Tim Wales who read and proofread all the background pages.
  • All the researchers, including undergraduate and postgraduate students, identified on the appropriate pages, who contributed to writing the Biographies.
  • Everyone, listed under the acknowledgements for the Old Bailey Project, who made possible the creation of the Old Bailey Proceedings Online, without which this project would not have been possible.
  • The Humanities Research Institute, University of Sheffield, its former director Professor David Shepherd, Michael Pidd, HRI Digital Manager, and Julie Banham, HRI Coordinator, for providing extensive technical and secretarial support for the project as well as a congenial home for the project staff.
  • The former Higher Education Digitisation Service, University of Hertfordshire, and in particular Asif Mohammed Farook and Ian Brearey, for managing the scanning and rekeying of documents; and Brian Robinson, former Director of HEDS.

Back to Top