Data and System Integration

ICT705 Data and System Integration – Task 2 Page 1 of 11
ICT705
Data and System Integration
Task 2
ATMC Semester 3, 2019
ICT705 Data and System Integration – Task 2 Page 2 of 11
Assessment and Submission Details
Marks: 40% of the Total Assessment for the Course
Due Date: 11:59pm Friday, 7
th February 2020 (Week 12)
Submit your assignment to Blackboard Task 2. Please follow the submission instructions in
Blackboard.
The assignment will be marked out of a total of 100 marks and forms 40% of the total
assessment for the course.
ALL assignments will be checked for plagiarism by SafeAssign
system provided by Blackboard automatically.
Refer to your Course Outline or the Course Web Site for a copy of the “Student Misconduct,
Plagiarism and Collusion” guidelines.
Late submission will be penalised according to the policy in the course outline. Please note
Saturday and Sunday are included in the count of days late.
Requests for an extension to an assignment
MUST be made to the course coordinator prior
to the date of submission and requests made on the day of submission or after the submission
date will only be considered in exceptional circumstances. Assignment submission extensions
will only be made using the official University guidelines.

ICT705 Data and System Integration – Task 2 Page 3 of 11
Background: Queensland Fire and Emergency Services
Given the recent outbreak of bush fires across the country the Queensland Fire and Emergency
Services (QFES) have noted deficiencies in their current systems. After evaluating various platforms,
the QFES has decided to adopt a Service Oriented Architecture (SOA) for its future IT infrastructure.
However, they would like to gain a deeper understanding of the technologies and have asked for a
demonstration of these technologies along with a brief explanation of the concepts and principles of
how it works.
Specifically, they have asked you to provide a demonstration web page in which the user will input a
region from a drop box and click a button to display a google map with the locations of fire stations
in the region.
Data for this assignment is based on data collected from Queensland Government websites,
specifically:
Station_Locations.xml was constructed from RuralFireStations.csv
Queensland Fire and Emergency Services – Rural Fire Stations dataset by Queensland Fire and
Emergency Services
from Open Data Portal (CC BY 4.0)
Fire_Stations.csv was constructed from data contained on web pages
Queensland Fire and Emergency Services – Regions by Queensland Fire and Emergency Services (CC
BY 4.0
)
Data from these two sources was downloaded and underwent some significant manipulation and
cleaning to produce the input files for your assignment, as working with the original data sources
was deemed too difficult for the course.
Station_Regions.csv was constructed consistent with the above data sources to help you complete
your assignment.
For those students looking for a challenge, you should try writing code that automatically extracts
and merges data from the original two sources, to experience a real-life example of the problems of
data integration even within one organisation (unfortunately no extra marks can be given for this).

ICT705 Data and System Integration – Task 2 Page 4 of 11
Assignment Tasks
This assignment consists of two deliverables, being: 1. Demonstration code 50%, and 2.
Report 50%.
1. Demonstration Code – All code relating to this assignment should be contained
within a folder named: Task 2 –
Your Name Student Number, the folder is then to
be zipped and uploaded to blackboard. (
note replace “Your Name” and “Student
Number
” above with your actual name and student number.
2. A report in Microsoft Word Format. The report must be uploaded separately.
Deliverable 1 – Demonstration Code
Important Note: For ease of demonstration, your code must be self-contained. In addition to using
Standards based HTML and Python, PETL, Bottle and Parser are the only additional frameworks you
should need. If you use others they must be provided and must not require installation.
To demonstrate your code, you have been provided with three data sources compiled with
data from QFES. These are:
“Fire_Stations.csv” contains the information about the fire stations,
“Station_Locations.xml” contains the location coordinates for each fire station
“Station_Regions.csv” contains a list of regions
You have also been supplied with two additional files, that you can use to check the
correctness of your implementation, or if you are unable to complete some tasks:
“Fire_Stations (ASCII).csv” contains data from Fire_Stations.csv with non-printable
characters removed
Fire_Station_Locations (Solution).csv” is the required output for Task 1
Task 1 – Data integration demo.
You are to submit a python script named “data_integration.py”.
Your script should clean and merge data into one CSV file named
Fire_Station_Locations.csv” with the following fields (attributes):

Field Name Mapping
RegionID Station_Regions.RegionID
Station Number Fire_Stations.Stn Number
Station Name Fire_Stations.Fire Station
Station Type Fire_Stations.Stn Type
Street Address Fire_Stations. Address
Phone Number Fire_Stations.Phone Number
Fax Number Fire_Stations.Fax Number
E-Mail New Calculated Field
Lat Station_Locations.Lat
Lon Station_Locations.Long

Table 1 – Field Mapping
ICT705 Data and System Integration – Task 2 Page 5 of 11
Task 1.1 Remove non-printable characters (5 marks)
The Fire_Stations.csv file contains non-printable Unicode characters, in order to perform other
cleansing tasks and merging of data you’ll first need to clean these characters from the data.
Note: You have been provided with a file “Fire_Stations (ASCII).csv”, if you are unable to complete
this task you may use this file to complete remaining tasks.
Task 1.2 Create E-Mail (4 marks)
An email address for each station needs to be constructed from its name: e.g. the address
for Acacia Ridge Fire Station will be [email protected]
Task 1.3 Merge Data (5 marks)
You are to merge data from the three sources into “Fire_Station_Locations.csv” with fields
as shown in
Table 1 – Field Mapping.
Task 1.4 Correct Field Names (2 Marks)
Ensure your field names match the fields as shown in Table 1 – Field Mapping .
Task 1.5 Correct Field Order (2 Marks)
Ensure the order of fields matches the order in Table 1 – Field Mapping
Task 1.6 Sort Data (2 Marks)
Data must be sorted firstly on “RegionID” and then on “Station Number”
Note: Should you be unable to fully complete the above tasks, you have been supplied with
a file “
Fire_Station_Locations (Solution).csv” that you can use for Task 2.
Task 1.7 Clean and Separate Street Address (Optional 10 Bonus Marks)
Separate data in the Address field into: “Street Address”, “Suburb” & “Post Code” fields. (5 marks)
Create a “State” field with a value of “Qld” for each record (2 marks)
Replace abbreviations for street type
(eg. Rd is used instead of Road, Crt instead of Court)
(3marks)
Note: This task is optional should you choose to attempt it any marks scored on this task will be
added to your overall mark for the assignment up to 100%, that is you can’t score more than 100%
for the assignment.

ICT705 Data and System Integration – Task 2 Page 6 of 11
Task 2 – RESTful Web Service Demo
You are to submit a python script named “data_services.py”.
For tasks 2.1 to 2.2 Your script should build a RESTful web service that supports a
‘getstations’ query from the client browser similar to “/getstations?regionid=x”. The server
Web Service will return a JSON table with the following fields (attributes): Station Number,
Station Name, Street Address, Phone Number, Email, Lat, Lon. Data is to come from the
“Fire_Station_Locations.csv” you built in task 1 (or, “Fire_Station_Locations (Solution).csv”
if you were unable to fully complete task 1).
Task 2.1 Return stations for a region (4 marks)
The server will return a table of records (tuples) of fire stations in the region passed to it.
Task 2.2 Return all fire station (3 marks)
In the advent of a call to “getstations?regionid=0”, the server will return a list of all fire
stations.
Task 2.3 List of Regions (3 marks)
To populate your drop-down list in task 3.1 you will need to return a JSON table of all
records from “Station_Regions.csv” with the following fields (attributes): RegionID, Region
Name. The server should respond to a “getregions” request.
Task 3 Mashup Demo
You are to submit a HTML file “stations_map.html” and optionally a CSS file named
“stations_map.css”. For ease of marking all JavaScript must be contained in your HTML file.
Task 3.1 Form Layout (5 marks)
You are to create an input form that accepts user input for a station region. The input for
the station region should be a drop-down list with the first item being “All Regions” with a
value = 0 remaining items should be created dynamically from a call to getregions from task
2.3. This input should be accompanied with a “Display Stations” button and appropriate
descriptors / information on the operation of the web page.
Task 3.2 Google Map (6 marks)
Clicking the “Display Stations” button will trigger a “getstations” web service to retrieve a
table of stations matching the region with attributes described in task 2. From this data you
are to display a Google map marking the position of the returned stations.
Task 3.3 Label Markers (2 marks)
Markers on the map should be labelled with the “Station Number”
ICT705 Data and System Integration – Task 2 Page 7 of 11
Figure 1 – Markers labelled with Station Numbers
Task 3.4 Centred & Zoomed (2 marks)
The map should be centred on the region and zoomed to an appropriate level to show all
fire stations in the region selected.
Task 3.5 Information Window Demographics (3 marks)
Upon clicking a marker bubble on the map, the user will be displayed an information
window containing: Station, Address, Phone and a clickable email address.
Figure 2 – Demographic Window
Task 3.6 Information window email (2 marks)
Clicking an email address link in the information window should launch the user’s email
application and automatically insert the subject line “Enquiry”.

ICT705 Data and System Integration – Task 2 Page 8 of 11
Deliverable 2 – Report
You are required to write a report explaining the key concepts around the design and
implementation of your demonstration system. You are also required to explain how to run
your code. Finally, you are to include all code submitted in the appendices to the report.
Your report should be no less than 1000 words and it would be best to be no longer than
1500 words. Note that the code will not count towards the word count. Your report should
follow the following template:
Executive Summary
Table of Contents
Table of Figures
(if needed)
Table of Tables (if needed)
1.0Introduction
2.0Key System Concepts
2.1 Data merging and cleaning
2.2 RESTful Web Services
2.3 Mashups
3.0Demo Running Instructions
4.0Conclusion & Recommendations
Reference List
Appendices
Appendix 1 – Data_integration.py
Appendix 2 – Data_Services.py
Appendix 3 – Stations_map.html
Executive Summary, Introduction, Conclusion & Recommendations (5 Marks)
Please follow the conventions detailed in:
Summers, J. & Smith, B., 2014,
Communication Skills Handbook, 4th Ed, Wiley, Australia.
2.1 Data Merging and Cleaning (10 marks)
Provide a scholarly definition of data merging and data cleaning.
What issues did you face and how did you deal with these?
What recommendations do you have for QFES regarding data integration and why?

ICT705 Data and System Integration – Task 2 Page 9 of 11
2.2 RESTful Web Services (10 marks)
Provide a scholarly definition of REST.
How was REST incorporated in your demo?
What problems can you identify with the design of your demo?
2.3 Mashups (10 marks)
Provide a scholarly definition of Mashups.
How were mashups incorporated into your demo?
How could you improve the system to provide a more efficient mashup?
3.0 Demo Running Instructions (10 marks)
Given your client QFES has limited technical knowledge, you are to provide instructions for getting
the ZIP file you supplied (the one you uploaded to BlackBoard) up and running. You should presume
that they have a new workstation with a clean installation of Windows to test your demo.
Report Format (5 marks)
The report must be formatted using the following guidelines:
Paragraph text 12-point Calibri single line spacing
Code Text 12-point Courier new single line spacing
Headings Arial in an appropriate type size
Margins 2.5cm on all sides
Title Page is not to contain headers, footers, or page numbering
Header is to display the Report Title (excluding the title page)
Footer to display your name and the page number (excluding the title page)
Title Page – Must not contain headers or footers. Include your name as the report’s
author.
Page numbering
o Executive summary to the last page of Table of Figures / Table of Tables to use
roman numerals (i, ii, iii, iv)
o Introduction and onwards to use conventional numerals (1, 2, 3, 4) starting at
page 1 from the introduction
The report is to be created as a single Microsoft Word document (version 2007, 2010,
2013, 2016, or Office 365). No other format is acceptable and doing so will result in
the deduction of marks.
The report must use the built-in features of Microsoft Word to create page numbers,
section numbers, citations, cross referencing of sections, figures and tables, table of
contents, table of figures, table of tables, and bibliography.

ICT705 Data and System Integration – Task 2 Page 10 of 11
Reference List (0 marks)
Please note that whilst there are no specific marks for referencing, it is expected, that as
masters’ students, that the source of all material is referenced. Please note the marks given
to sections 2.1 to 2.3 will be based on the quality of your references, and that failure to
reference material will result in few if any marks and potentially lead to a charge of academic
misconduct. Please also note you are to submit a Reference List not a Bibliography, thus only
references citated in your report should appear in your reference list.
References should follow the Harvard method of referencing. Note that ALL references should
be from journal articles, conference papers, technical papers or a recognized expert in the
field. DO NOT use Wikipedia as a reference. The use of unqualified references will result in
the deduction of marks. The exception to this is where you have copied code or based your
code on a source in which case make a comment in your code acknowledging the source and
insert a citation in your report (the quality of the source is not important, if you use it,
reference it).
Appendices (0 marks)
You are to submit, in text format, the code you have written for data_integration.py,
data_services.py, stations_map.html and stations_map.css (if applicable).
NOTE: failure to include your code will result in ZERO marks being awarded for
Deliverable 1 – Demonstration Code, and likely result in an investigation into
Suspected Academic Misconduct.
Assignment Return and Release of Grades
Assignment grades will be available on the blackboard in two weeks after the submission. An
electronic assignment marking sheet will be available at this time. Where an assignment is
undergoing investigation for alleged plagiarism or collusion the grade for the assignment and
the assignment will be withheld until the investigation has concluded.

ICT705 Data and System Integration – Task 2 Page 11 of 11
Assignment Advice
This assignment will take several weeks to complete and will require a good
understanding of application and information integration technologies for successful
completion. It is imperative that students take heed of the following points in relation to
doing this assignment:
1. Ensure that you clearly understand the requirements for the assignment – what
must be done and what are the deliverables.
2. If you do not understand any of the assignment requirements – Please
ASK
your tutor or course coordinator if your tutor cannot resolve your problem.
3. Each time you work on any aspect of the assignment reread the assignment
requirements to ensure that what is required is clearly understood.
4. Prior to submitting your code, you should ensure not only that it executes as
required, but also looks professional. It is expected that you adhere to HTML and
python standards for naming and indenting. All methods should be adequately
documented such that another programmer examining your code will readily
know what the code is doing.
Plagiarism and Collusion Advice
1. All work must be submitted through SafeAssign.
2. SafeAssign will pick up any similarities between work online as well as work from
other students (in this semester and previous)
3. Please make sure you reference your work properly. If you are using any material
from the internet or any books from the library, you need to cite the work
correctly. Failure to do so will result in possible cases of Academic Misconduct.
4. Please do not share your work with other students. Do not give anyone your files
to have a look. SafeAssign will pick up collusion, but keep in mind the
percentages for Collusion may not report accurately until all student assignments
have been submitted. Both the person copying and the person providing will
potentially be held accountable.
5. You can submit a draft assignment through SafeAssign before making the actual
submission.
6. If you need any advice or are unsure about referencing, please speak with ATMC
Administration for assistance.