How to Bulk Upload Observations & Data

Data may be uploaded to FieldScope in two ways. While the most common way is for participants to upload data one observation event at a time (see Adding Data to a Project for more on this), project organizers may also use the bulk upload option. The bulk upload FieldScope tool allows project organizers, or individuals designated by project organizers, to save time by uploading large sets of data in a spreadsheet format

In this article, we cover how to use the bulk upload tool, including formatting your data spreadsheets correctly. 

The bulk upload feature is only available to people set as a manager of a FieldScope project, or specially designated participants in a project. If you are a member and not a manager of a project and need to bulk upload data, contact us. 

Steps to upload

1. Decide if you will need 1 or 2 spreadsheets.

Most commonly, uploaders will need to upload an observation and a station location spreadsheet that are linked together through a common Station ID. But there are scenarios where uploaders may only need to upload one type of spreadsheet or the other. 

             If you need to:

• • Upload new observations to new stations locations, then upload an observation AND station spreadsheet (2 total)
  • Upload new observations to existing station locations already added to the FieldScope project, then upload an observation spreadsheet only (1 total)
  • Upload ONLY station locations upload a station location spreadsheet only (1 total)

2. Format your spreadsheets.

Observation spreadsheets must include a header row where the data you intend to upload is represented in each column. There are two required columns:  Observation Date - the date when the data was collected Station ID - to link it to locations already in FieldScope or in your station location spreadsheet If the stations are already included in FieldScope, you will need to include the unique Station ID identifier assigned by FieldScope. More on that below.  Any other columns are optional and depend on the data you have collected. In some cases, a project's protocol may include more data fields that you have collected. You only need to include columns for each data field you are submitting.  Below is an example of how an observation spreadsheet might look with the two required columns.

Station spreadsheets must also include a header row, and the following are required columns: Latitude - formatted in decimal degrees Longitude - formatted in decimal degrees Station Name - the name you want to designate to this location Station ID - If uploading new stations, you can use any ID (e.g., 1, 2, 3, 4, 5…) that you will also use in your observation spreadsheet to link observations to station locations.  Some projects may have additional station fields as part of their protocol that can be optionally included. See your project’s protocol for more.  Below is an example of how a station spreadsheet might look with the 4 required columns.

Note on units on numeric fields: At this time, the bulk upload tool does not allow you to select units. If you are uploading data to a numeric field that has multiple unit options in FieldScope, you should make sure to use the default unit set for that FieldScope project and convert any data you have in other units as needed. You can see the default unit by noticing which unit is the first option when entering data one observation at a time.

3. Initiate a spreadsheet upload.

From the main project page, click on the three dots next to the “+ Data” button in the upper left corner of your browser window and select Upload from the dropdown menu.

4. Select spreadsheet files.

After you select an option, you will receive a pop-up window allowing you to select the CSV or Excel from your computer. Once you select a file, you will see a screen that looks like the one below.

5. Match fields from spreadsheet to fields from database.

The list of field names on the left represent all of the database fields for the project you are uploading to. To the right is a dropdown list of all the column headings in your spreadsheet. If your column headings exactly match the names of the fields in the database, they will populate automatically. If not, you should go through and do the matching yourself. Fields that have been successfully matched will have a green checkbox indicating that that data from your spreadsheet will be included in the upload.

6. Initiate upload.

Once you are satisfied that all of the fields you are uploading are pulling from your file correctly, click on the Upload button. Please note: Only click on the Upload button once. Pressing it multiple times may result in inaccurate data uploading to the platform.

7. Fix any errors, do a final review, and commit your upload.

If a file has flawed data or station information, you will receive an error message. See tips on dealing with errors and example errors below. If there are no errors showing, you will see the screen below. Complete a final check of the data to ensure everything uploaded correctly; then click on Commit to complete the process. Please note: Only click on the Commit button once. Pressing it multiple times may result in inaccurate data uploading to the platform. If you suspect the platform has not accepted the Commit command, wait a few minutes, open a new browser window, and check to see if the observations you tried to load are visible. If not, you can refresh and try the upload option again. An excess of data points in spreadsheets can cause problems with the upload process. Please make sure you are uploading no more than 5,000 rows of observation data at one time.

You will receive a notification at the top of your browser window that the bulk upload is complete.

Addressing Errors in Your Spreadsheet

Common errors include:

• Including special characters or letters in a field that is set as numeric (e.g., including the word “good” in a column set for numeric water temperature data)
• Misspellings or the incorrect format of words in a field that is expecting values from a set list of values (e.g., if a field in the project you are uploading to is expecting either “Yes” or “No” as options, but you include “I’m not sure” as a value)
• Data not being formatted into columns and rows correctly
• Spreadsheets not being saved as CSV or Excel files

If you are uploading observations to station locations already included in FieldScope, make sure to include the unique 7-digit ID generated by FieldScope in the Station ID column. More on how to look up the unique ID for a station below .

In the example below, the stations (locations where data were collected) listed in the file were not found in the FieldScope platform. These stations would need to be added to the platform in order for the bulk upload tool to accept the file for upload, so you should upload a separate Stations spreadsheet at the same time.

You will also receive an error if the file you upload includes data in fields that do not allow for those kinds of data. In the example below, there is a letter in a field that only allows for numeric data. Note that you can use the scroll bar to see exactly where the erroneous data, listed in red, are located in your file. Remove or update the incorrect data in your file and then try to upload again.

After you have addressed any errors, click on the Clear button so you can re-upload the file.

Finding the Unique Station ID 

Please note: This step only applies to projects that use stations. Projects with stationless data can skip this step when bulk uploading.

FieldScope assigns every station location its own unique ID. If you are uploading new stations to the platform, make sure you keep track of the unique ID the platform assigns to your stations, so you can tie later observations to that location. Later on, when you are uploading an observation spreadsheet to locations that are already included in FieldScope, you will need to include the unique ID generated by FieldScope in the station ID column of your observation spreadsheet. 

To find the unique ID of a station, open any observation from that station in edit mode and the unique Station ID will be displayed (for more on this see the Editing Data article). 

Note that only project contributors that created a station or managers in a project (or group coordinators in projects with groups or chapters, like FrogWatch) can access the unique station ID.