arrow-left

All pages
gitbookPowered by GitBook
1 of 12

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Quick Review: Spatial Data

Spatial data adds a geographic dimension to a qualitative and/or quantitative data set, situating it in a particular location within a coordinate system relative to other data points. (The coordinate system can be a real-world system or a locally created one used to meet the needs of a particular project.)

Spatial datasets, in general, come in two distinct forms, vector dataarrow-up-right (points, lines, and polygons) and raster (or pixel data)arrow-up-right. Raster and vector data can come together in the creation of a wide variety of mapping projects, from a traditional figure with an explanatory legend and caption, such as might appear in an academic text, to an online interactive platform that allows for the searching or filtering of thousands of pieces of spatial data or hundreds of historical maps.

hashtag
Vector Data

Vector data includes points, lines, or polygons (shapes made up of straight lines) containing spatial information that represent some sort of feature or event in a physical or imagined landscape and may contain other types of qualitative or quantitative information, called attributes. A point may represent a tree, a city, or a moment in time. Lines might indicate the street grid of a town, the path someone traveled across the world, or a social link between two communities. Polygons can mark the boundaries of a country or voting district, the catchment area of a river, or a single city block.

For example, the relatively simple and ongoing project from the Burn's Library collection pictured below uses vector point data to offer a selection of images and accounts from individuals and their observations about how the cities and landscapes they visited appeared. Users can filter the point data by data or search for particular location names in the search bar.

hashtag
Raster Data

Raster consists of "cells" of data covering a specific area (its extent), with attribute values in each cell representing a particular characteristic. It may still consist of points, lines, and polygons, but these shapes are themselves composed of pixels (the way a jpeg or other image file type is).

Data of this type may take many forms, such as satellite imagery containing vegetation or elevation data, precipitation maps, or even an historical map, which has been given a spatial reference. Unlike vector data, raster data has a particular resolution, meaning each pixel represents a particular geographic region of a specific size.

Most projects combine various forms of vector and raster datasets.

World Travel and Descriptionarrow-up-right
Screenshot of World Travel and Description
Vector vs Raster data

Workshop: Exploring and Creating Vector Data

An opportunity to explore one of the spatial databases discussed in the presentation to look more closely at how their data is organized and to create your own simple spatial dataset.

Create a few spatial points of interest using the format we saw above ObjectID, X (long), Y(lat), or any other attributes you want using GoogleSheets/Excel/OpenOffice; or explore one of the databases above and practice downloading and opening the datasets to get comfortable dealing with vector datasets.

Remember to save your file as a .csv when you are done!

Some Boston Spatial Databases:

hashtag
Possible Datasets

Possibly useful datasets for exploring questions about segregation and redlining.

  • (download CSV)

  • (download Shapefile)

hashtag
Bonus

Want to check your csv data real quick? Go to , create a New Map, and click "Import" to import your CSV data to make sure it's looking good. Otherwise, we will be checking it later when we get to ArcGIS online!

An Introduction to Mapping, GIS and Vector Data

Boston Social Vulnerability arrow-up-right(download Shapefile) [play with attribute and drawing style options]

  • Public Schoolsarrow-up-right (download CSV)

  • Non-Public Schoolsarrow-up-right (download CSV)

  • Analyze Bostonarrow-up-right
    MBTA Open Data Portalarrow-up-right
    Mapping Inequality in New Deal Americaarrow-up-right
    Hospitalsarrow-up-right
    Neighborhood boundariesarrow-up-right
    Farmer Marketsarrow-up-right
    Google My Mapsarrow-up-right

    Tutorial: Georectifying Historical Maps using MapWarper

    Map Warperarrow-up-right is an open source map warper/map georectifier, and image georeferencer tool developed, hosted, and maintained by Tim Waters.

    hashtag
    Creating an Account

    In Map Warper, it is possible to browse and download maps others have uploaded into Map Warper without an account. To georectify your own map, however, you must make one. This also allows you to easily return to your maps later.

    circle-info

    All you need to create an account is an active email address. It may also be linked to an active Facebook or Github account.

    1. On the top right corner of the page, click "Create Account"

    2. Select a username and password and enter an active email address.

    3. Click "Sign up"! You should quickly receive an email to confirm your account

    hashtag
    Upload a Map with any Available Metadata

    Now that you are logged in, you can upload your own images to the Map Warper server in order to georeference them.

    circle-exclamation

    By uploading images to the website, you agree that you have permission to do so, and accept that anyone else can potentially view and use them, including changing control points. As a freely available tool, you should not expect Map Warper to store your map indefinitely; once it has been georeferenced, you should plan on storing your georeferenced map on your local hard drive or a file storage platform like GoogleDrive.

    1. Clicking “Upload Map” on the main toolbar (note if you are not yet logged in, it will ask you to do so at this point)

    2. Insert any available metadata and a description of the map. This is useful both for your own records and for anyone else searching for similar maps on the Map Warper server.

    3. At the bottom of the page, choose to Upload an Image File from your local computer or from a URL. Once the file has been selected, click "Create"

    Once the upload is complete, a new page will appear informing you that the map was successfully created as well as providing an image of the uploaded map.

    hashtag
    Rectify a Map with Control Points

    Now the map is on the platform, but it does not yet have any spatial information associated with it. The next step is to use what are called "control points" to place your map in a “real-world” coordinate system where it can interact with other types of spatial data.

    circle-info

    Note that you can also edit the original metadata fields, crop out unwanted portions of your map, and see a history of the interactions with the map at this point from the main toolbar

    hashtag
    Understanding the Georectifying Window

    1. Once your image is displayed, select "Rectify" on the main toolbar.

    2. This opens up the Georectifying page, the most important page in this tutorial. It is composed of two windows, one showing your map and one showing a “basemap” which you will be using to geolocate your map.

    3. In the top right corner of each map there are a series of buttons that help you navigate the map and add control points

    Add a control point to the maps

    Move a control point that has already been added to the map

    Pan around the map

    The last two icons appear only on the basemap and are used to adjust it as needed to help with georeferencing

    Change the basemap to a Custom, already georectified basemap of your choosing (if available)

    Swap between the initial basemap and satellite imagery, depending on which is easier to georeference your map

    hashtag
    Creating Control Points on your maps

    1. Navigate on your map to an easily identifiable location. In this example, I have chosen the tip of the island in the middle of Paris that the Notre Dame Cathedral is on. Note that an external mouse with a scroll wheel can make the zooming/moving process easier; zoom and pan buttons are also provided in each window.

    2. Click the “Add Control Point” icon, then click again on your map in the desired location. A little control point should pop up!

    3. Swap to the basemap and click the “Pan” tool (the hand) to find the proper location, then again select the “Add Control Point” tool and click on the corresponding point on the Basemap.

    circle-exclamation

    You will need at least 3 control points to geolocate your map, but more is preferrable. It is also advisable to spread your points across the map rather than have them clustered; this will ensure that the map is georeferenced equally across the map rather than only in one area. If you need to delete a point, this option is available from the "Control Points" table

    circle-info

    Remember: places change over time! Try to use features that remain as consistent as possible on both maps. In general, the more control points you add, the more accurate your map will be.

    hashtag
    Check for Error and Warp the Map

    1. After you add the 4th control point, your table of points will start including error information, as the points are triangulated against one another. Note that this error may not mean that you are doing anything wrong, particularly in an older map that is not as spatially accurate as something more modern! On the other hand, if your error is quite high and you believe your map is relatively accurate, you may have misplaced a control point somewhere. Usually, high error is caused by a single point being misreferenced.

    2. When you feel like you have enough points scattered around your map, we are ready to georectify the map! Remember you can always come back later and add new points or remove old ones if you feel like the result is not to your liking. To georectify your map, just click “Warp Image!” at the bottom of the page and you’ll get a notice that your rectifier is working.

    When the map is finished rectifying, you will get a notification that rectification is complete. Now, you should be able to see your map overlaid on the Basemap, as well as be able to turn it on and off or more or less opaque to check for accuracy!

    hashtag
    Exporting your Map

    If the map is to your liking, you are ready to export. Map Warper offers a variety of ways to export your map depending on your needs

    1. To export your map, Select the Export tab on the toolbar. A window like that seen below will pop up, giving you a variety of choices for exporting

    Some exporting options:

    • GeoTiff:

      • public domain standard; easily imported into a wide variety of spatial platforms like ArcGIS or QGIS; good for backing up your georeferenced map on your local computer or in cloud storage like Google Drive

    • .kml:

    circle-exclamation

    Adding your tiles to an ArcGIS online map can be complicated. From an empty map, choose Add --> Add Layer from Web and then select a "Tile Layer". Where it says “URL” copy over the Tiles (Google/OSM scheme) URL from your Map Warper file. It will look something like: .

    However, note that the end of the URL should look like “{level}/{col}/{row}.jpg” according to the instructions given. Replace the at the end of your URL with this ending, creating something that looks like: . It should now load properly into ArcGIS online

    Adding Base-maps and Raster Data

    When you load up a new map, everything is blank...lets get something to start with

    Initially, QGIS only has a single basemap to work with, let's add it!

    In your browser pane, find XYZ Tiles and click the down arrow to find the Open Street Map basemap. Click and drag it into your layers box, and it will appear on the map.

    There are lots of different options for loading basemaps, but first you have to connect them to your project from their hosted locations online (or self host). Here's a fast way to get a bunch of standard basemaps though, thanks to Klas Karlsson (one of the main qgis devs)!

    1) Download the python script below and open it in a text editor (e.g. notepad).

    Adding and Creating Basic Vector Data

    Lines, Points, and Polygons

    Ok, we've got a background map, now what about vector data?

    hashtag
    Adding Premade Vector Data

    Adding vector data depends on what kind of format your data is in. Let's run through the standard three types.

    The goal here is to create what are called, “control points,” or points that are corresponding between your uploaded map and the basemap. This is done by simply zooming in on each map in turn and creating a control point as close to the same point as possible in each map.

    Once you have created a control point on each map, scroll down and click the “Add Control Point.” This will add the control point coordinates to a list of points below, which you can see by clicking the words “Control Points."

    Easy import directly into GoogleEarth

  • Tiles (Google/OSM scheme)

    • Useful for loading into tools like ArcGIS online and Knightlab StoryMap JS. Remember to ensure a backup of your files elsewhere though in case your map is eventually removed from Map Warper.

  • https://mapwarper.net/maps/tile/49503/{z}/{x}/{y}.pngarrow-up-right
    {z}/{x}/{y}.pngarrow-up-right
    https://mapwarper.net/maps/tile/49503/{level}/{col}/{row}.jpgarrow-up-right
    Choose "Create Account"...
    ...and enter in a username, password, and active email address!
    Once you are logged in, select "Upload Map"...
    ...enter any available metadata...
    ...and then choose what image you want to georeference, and click "Create"!
    Georectifying page, with your map on the left and a basemap on the right...
    ...and each has a series of icons in the top right corner
    Click the Control Point icon then choose a location on your map...
    ...and repeat this process on the basemap at the same location!
    Finally, click "Add Control Point" to set the two points equal to one another.
    Error is shown in the Control Points toolbar, color-coded by suggested severity.
    When you are ready, hit "WARP IMAGE!"
    A pretty accurate map of central Paris!
    Lots of options for exporting
    2) Open up the Python console in QGIS by going to Plugins-->Python Console

    3) Copy the python script into the editor and press enter.

    4) Enjoy all your basemaps! Thanks Karl!

    What if you want to load your own georeferenced map, like one from MapWarper? Easy enough!

    1) Right-click on XYZ and select "New Connection"

    2) Look back at your mapwarper project page and choose the Tiles URL (the same one you use with ArcGIS online or Knightlab Storymaps

    The Mapwarper Export Page

    3) Name your connection and you should be good to go!

    Name and URL for your hosted map
    We've successfully imported a historical map!

    Finally, what about importing a local raster file, whether its a georeferenced historical map, a Digital Elevation Model, or some other raster. Even easier!

    1) Make sure you know where the file is hosted on your computer, or download the file from the internet (Like with the rectified GeoTiff from Mapwarper)

    2) In the main toolbar, go to Layer --->Add Layer-->Add Raster Layer

    3) In the Source section, click the "..." and navigate to your saved georeferenced raster as your raster dataset, open it up, and click "Add" to add it to your map.

    It may look the same, but this map is locally hosted!

    That's it for the basics of adding rasters and basemaps to QGIS!

    file-download
    8KB
    basemaps.py
    arrow-up-right-from-squareOpen
    Thanks Klas!
    Now we have a basemap to start from!
    hashtag
    CSV

    To add CSV data to the map, go to Layer-->Add Layer --> Add Delimited Text Layer

    Here I've selected a csv of all the places mentioned in a Jesuit Catalogue (about 30000)arrow-up-right

    Note the sample data at the bottom of the window

    And there you go!

    All the Locations Mentioned in the Jesuit Project

    hashtag
    Shapefiles

    Shapefiles are a very common file type for individaul spatial data layers and can be downloaded from spatial database sites like BostonMapsarrow-up-right. To import a shapefile into QGIS:

    1) Go to Layers-->Add Layer-->Vector Layer

    2) Select your ZIPPED folder containing your shapefile as the file to be imported, and click Add!

    Select your zipped shapefile

    And your shapefile (in this case polygons representing the open, public spaces of Boston, is added to the map!

    Polygon Shapefile

    hashtag
    GeoJson

    GeoJson files are imported the same way as Shapefiles, though no need to zip since its just one file!

    hashtag
    Creating your own Vector Data

    Up to now, we've been adding other people's data, or data we have generated in a spreadsheet....what if we want to creat our own data from scratch inside QGIS?

    Instead of Adding a Layer, we Create a new one!

    Create a new Shapefile from Scratch

    When making a new vector layer from scratch, you define its type and attributes similar to how you would in a spreadsheet.

    You've got the power!

    Here I've created a Point shapefile called "Cities". To start adding to it, I just right-click and toggle editing on!

    When editing is on, the small editing toolbar will become accessible

    From left to right the tools are:

    1. Current Edits allows you to manage your editing session. Here you can save and rollback edits for one or more selected layers.

    2. Toggle Editing provides an additional means of beginning or ending an editing session for a selected layer.

    3. Save Layer Edits allows you to save edits for the selected layer(s) during an editing session.

    4. The Add Features tool will change to the appropriate geometry depending on whether a point, line or polygon layer is selected. Points and vertices of lines and polygons are created by left clicking. To complete a line or polygon feature right click. After adding a feature you will be prompted to enter the attributes.

    5. Features can be moved with the Move Tool by clicking them and dragging them to the new position. Individual feature vertices can be moved with the Node Tool. Click on a feature once with the tool to select it, the vertices will change to red boxes. Click again on an individual vertex to select it. The selected vertex will turn to a dark blue box. From there the vertex can be moved to the desired location. Additionally, edges between vertices can be selected and moved. To add vertices to a feature, simply double click on the edge where you want the vertex to be added. Selected vertices can be deleted by clicking the Delete key on the keyboard. Features can be deleted, cut, copied and pasted with the Delete Selected, Cut Features, Copy Features, and Paste Features.

    After clicking to create a new Feature, a box to fill in the attributes you've chosen will appear. Below are the attributes for my pretend Cities shapefile.

    You can access and edit all the attributes from your current shapefile by right clicking the name of your layer and selecting Open Attribute Table.

    The two points in my current spreadsheet.

    Styling your data and preparing it for exporting

    Now your data is in....how to make it look good?

    hashtag
    Styling your data

    Let's look at different ways to style your data using some archaeological data from the site of Gabii originally imported as a .geojson.

    When initially imported, all the polygons are red with a black outline, which gives us some general information about the location but doesn't really differentiate the different kinds of archaeological features present. To do that, we need to change our styling.

    Initial Import

    To get to the styling options (also called Symbology), just double click the layer name in your Layers pane.

    Here you can select how you want your features to be styled; the default is "Single Symbol" but often times you will want Categorized, or for quantitative attibutes, Graduated, ways of displaying your features. For now, let's select Categorized.

    When stylizing by category, you can select which Attribute(s) you want to stylize by; I'm going to select the Descriptio(n) field because that's the different types of features in my layer.

    Once you've picked your attribute, clicking "Classify" at the bottom of the box will assign an initial style to each of your features.

    Each feature type is now a different color! You could further customize each individual style if you wanted on the symbology page.

    Or, here I have made the color of the outline of the feature change, rather than the fill, in order to make the map more understandable.

    Lots can be done with symbology; feel free to explore!

    hashtag
    Preparing to Export Your Figure

    Exporting your figure to share with others or for publication is often the final product for your QGIS map. Here we talk about the basics.

    So far we have been working in what I call "Data View" where you can create, edit, and generally mess with your data. Now we are going to look at Print Layout, which allows you to organize your map for publication.

    To access print layout, go to Project-->New Print Layout. A box will appear asking for you to name the layout (useful if you are creating mulitiple figures, for example).

    A new window will appear that lets you create your layout, almost like working in a word document.

    The left toolbar is your friend, allowing you to add map windows to the figure, as well as things like maps, north arrows, titles, legends, and other labels.

    Each time you add a feature, it will appear in what is basically the "Layers" sidebar of your print layout, allowing you to further edit its properties. Above I've quickly added a scale bar, north arrow, and legend to the map.

    When it looks good, you can export the figure! My figure below is a .tiff

    Symbology Menu
    Ways to Symbolize
    Select your Attribute
    https://docs.google.com/presentation/d/1wX3GIZpNF1ZFXJu950Hd3gmq9PhJTIndOBbiNCazX1I/edit#slide=id.g1310d00011c_0_890docs.google.comchevron-right

    Mapping and GIS

    This portion of the incubator is focused on an introduction to spatial data (vector and raster), with workshops on creating spatial data and finding and georeferencing historical maps. Finally, the different datasets are combined in an ArcGIS online map.

    1. An Introduction to Mapping, GIS, and Vector Data

    2. An Introduction to Raster Data and Georeferencing Historical Maps

    Bonus: QGIS Workshop (in progress)

    An Introduction to ArcGIS Online

    Presentation + Workshop: Putting it together in ArcGIS Online

    Workshop: A Brief Introduction to QGIS

    QGIS is a powerful open-source Geographic Information System platform with a bit of a learning curve...

    Welcome to the Wild World of Desktop GIS tools

    QGIS (and its assoicate ArcGIS Pro/Desktop) are powerful tools used for a wide variety of purposes across many fields. And GIS in general is a huge subject, with entire degrees/certificate programs devoted to it. As such, we will barely scratch the surface in this workshop, but by the end you will be able to:

    1. Understand a bit about the QGIS interface;arrow-up-right

    2. Search for and add pre-created basemaps;

    3. Add your own georeferenced raster data;

    4. Import basic vector datasets from csvs, shapefiles, and geojsons;

    5. Create a new vector dataset from scratch;

    6. Perform some basic styling of data;

    7. Prepare your data for export.

    That's a lot....lets get started!

    For those interested in diving deeper, I highly reccomend reading the thorough ; much of the below is taken from there and the step-by-step Links to specific parts of the handbook are referenced above.

    QGIS Handbookarrow-up-right
    Training Manual.arrow-up-right

    Workshop: Finding and Georeferencing an Historical Map

    Explore one of the databases discussed in the presentation and download an historical map of interest (or use one of your own if you already have one!). Then, using the MapWarper online tool, georeference your map with at least 4 points.

    hashtag
    Possible Mapping Geodatabases

    • David Rumsey Map Collection ()

    • Library of Congress Map Collection ()

    • USGS Historical Topographic Map Explorer ()

    hashtag
    Redlining Map Option

    • To look at your data against an historical Boston redlining map, we recommend using one of the from the Boston Public library.

    Georeferenced version of the redlining map:

    https://www.davidrumsey.com/arrow-up-right
    https://www.loc.gov/maps/arrow-up-right
    https://livingatlas.arcgis.com/topoexplorer/index.htmlarrow-up-right
    "Residential Security Maps"arrow-up-right
    https://mapwarper.net/maps/tile/65580/{z}/{x}/{y}.png

    An Introduction to Raster Data and Georeferencing Historical Maps

    https://docs.google.com/presentation/d/1BXYzvfOhhGSgK-41xtr8FvMpm3Cf-0Lo_8vjLXW1aEE/edit#slide=id.g1310dbaabf3_0_0docs.google.comchevron-right
    https://docs.google.com/presentation/d/1Afbpd6PorjoiqS810hoQrQBtyQXRyY1h115QGn3GHno/edit?usp=sharingdocs.google.comchevron-right