Predictive Analytics

MIS772 Predictive Analytics – Assignment A1 / Rubrics for LP1 and LP2
1 of 1
Assignment A1
must use the provided template. It will be assessed as follows.

LP1+2 Exceptional Meets Expectations Improve
Exceptional Very Good Meets
Expectation
Acceptable Improve Unaccpt
10 8 6 4 2 0
Exec Report
LP1+LP2/ULO1
LP1: Business decisions and actions
that the analytic solution could support
are well explained.
LP1: A business problem (or question) is
stated succinctly in business terms.
LP1: Problem not
described.
LP2: Business decisions and actions
are clearly supported by the analytic
solution. Any
academic references
cited throughout this report may be
included in this section.
LP2: The business solution succinctly
described for executives and justified.
Cross-references with the technical
sections of the report provided for
support, e.g. to tables, charts and plots.
LP2: Solution not
described or justified.
Over the page limit.
10 8 6 4 2 0
Data Features
LP1+LP2 / ULO1
LP2: All missing values and data
errors
handled adequately. Results
are comprehensive and tabulated.
Answer to business question (A)
given and justified.
LP1: Data obtained. RM project prepared.
Important attribute properties
(numerical
and nominal, i.e. non-text) are tabulated,
analysed and reported. Characteristics of
the selected attributes explored using
charts and tables. All charts annotated
(with text and arrows).
Not meeting
expectations.
RM not used.
Missing RMP files.
Over the page limit.
W2=20* 10 8 6 4 2 0
Data Rels
LP2/ULO1
Attribute weights are used to
determine what attributes are useful
for prediction. Selection of best
attributes is made and justified.
Attributes are transformed or
generated as needed.
Relationships between attributes
(numerical and nominal, i.e. non-text),
are explored and visualised. Selection of
labels and predictors recommended and
justified. All charts annotated (with text
and arrows) to highlight insights.
Not meeting
expectations.
RM not used.
Missing RMP files.
Over the page limit.
W3=30 20 16 12  4 0
Models
LP2/ULO2
Decision Tree included as the second
classification model. The process, its
operators and parameters described
and justified (why).
Class imbalance
investigated, dealt with, justified.
k-NN classification model included. The
analytic process, its operators and their
parameters described and annotated
(with text and arrows). The values of the
model parameters justified (why).
Not meeting
expectations.
RM not used.
Missing RMP files.
Over the page limit.
20 16 12  4 0
Evaluation
LP2/ULO2
Models are cross-validated and tested
(hold-out not required), performance
tabulated and compared
– the best
model identified. In addition to
accuracy and kappa measures such
as
TPR, FPR, AUC, and ROC, are
also used (as applicable).
The model is hold-out validated and
tested using accuracy and kappa. All
validation and test results are analysed
and reported. A statement is included
with justification of whether or not the
model is acceptable and can be trusted.
Not meeting
expectations.
RM not used.
Missing RMP files.
Over the page limit.
W4=70 15 12 9 6 3 0
Deploym.
LP2/ULO2
Answer to business question (C)
given and justified.
Step-by-step explanation given on how to
execute the process and replicate results,
e.g. or when applied to new data. Screen
shots must be included for illustration.
Answer to business question (B) given
and justified.
Not meeting
expectations.
RM not used.
Missing RMP files.
Over the page limit.
W5=85* 15 12 9 6 3 0
Research
LP2/ULO2
New and surprising insights from data
reported. Additional readings cited
throughout the report, their list is to be
included in the executive section.
New analytic methods used (2-3) for data
analysis, modelling or visualisation –
beyond what was covered in class. You
can use RapidMiner, but also R, Python,
or some other tool (for this section only).
Not meeting
expectations.
Missing RMP or
script files. Over the
page limit.
Extras Not MS Excel!

Submit weekly: Report (use template, in PDF format) and RapidMiner process files (RMP files in ZIP archive).
Missing report, RMP files, or their incorrect format will disqualify the submission!
Late penalties will be calculated based on the date and time of the last submitted component.
Assessment
Criteria
Weekly
submissions
are suggested
and the
estimated
completion
indicated in
red on the
margin
LP1 and LP2
submitted
by their
respective
deadlines
indicated with
a star in week
W2 and W5
Extensions
will not be
granted
unless
weekly
progress
submissions
are made!