Description

Course: Algorithms on Graphs (Course 3 out of 6)
Specialization: Data Structures and Algorithms
Programming Project:
Advanced Shortest Paths
Introduction
We also encourage you to create a forum thread and compete whose solution to the problem “Contraction Hierarchies Large” is the fastest in terms of both preprocessing time and query time, and also which solution is the most memory efficient. You can post the screenshots of the grader feedback in the comments, and the thread owner can keep updated the table with the top results in different nominations in the initial post itself.
The grader will show you the input data in the coding problems only if your solution fails on one of the first few tests (please review the questions 7.4 and 7.5 in the FAQ section for a more detailed explanation of this behavior of the grader). Also, in the problems with preprocessing phase (starting from the third problem, “Compute Distance with Preprocessing”), there are two separate inputs — the graph itself (for preprocessing) and the queries for computing the distance. You will be shown both inputs in case your solution fails on one of the first few tests. If your solution fails on further tests, you won’t see the inputs.
Learning Outcomes
Upon completing this programming assignment you will be able to:
1. implement Bidirectional Dijkstra and use it to compute distances in social networks very quickly;
2. implement A* search algorithm and apply it to compute distances in road networks faster than the classic algorithms do;
3. implement Contraction Hierarchies algorithm and apply it to preprocess large road networks and then compute distances in them much faster;
4. implement even more heuristics speeding up Contraction Hierarchies and solve even bigger road networks;
5. use your Contraction Hierarchies implementation to go further and solve the classical logistics problem (called Travelling Salesman Problem): find the optimal path for a truck leaving the depot to visit a list of stores, deliver the goods to them and return to the depot.
Passing Criteria: 3 out of 5
Passing this programming assignment requires passing at least 3 out of 5 code problems from this assignment. In turn, passing a code problem requires implementing a solution that passes all the tests for this problem in the grader and does so under the time and memory limits specified in the problem statement.
Contents
1 Problem: Friend Suggestion 3
2 Problem: Compute Distance Faster Using Coordinates 6
3 Problem: Compute Distance with Preprocessing 9
4 Advanced Problem: Compute Distance with Preprocessing on Larger Road Networks 12
5 Advanced Problem: Travelling Salesman Problem 14
6 General Instructions and Recommendations on Solving Algorithmic Problems 17
6.1 Reading the Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.2 Designing an Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.3 Implementing Your Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.4 Compiling Your Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.5 Testing Your Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.6 Submitting Your Program to the Grading System . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.7 Debugging and Stress Testing Your Program . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7 Frequently Asked Questions 20
7.1 I submit the program, but nothing happens. Why? . . . . . . . . . . . . . . . . . . . . . . . . 20 7.2 I submit the solution only for one problem, but all the problems in the assignment are graded.
Why? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.3 What are the possible grading outcomes, and how to read them? . . . . . . . . . . . . . . . . 20
7.4 How to understand why my program fails and to fix it? . . . . . . . . . . . . . . . . . . . . . 21
7.5 Why do you hide the test on which my program fails? . . . . . . . . . . . . . . . . . . . . . . 21
7.7 My implementation always fails in the grader, though I already tested and stress tested it a
lot. Would not it be better if you give me a solution to this problem or at least the test cases
that you use? I will then be able to fix my code and will learn how to avoid making mistakes.
Otherwise, I do not feel that I learn anything from solving this problem. I am just stuck. . . . 22
1 Problem: Friend Suggestion
Problem Introduction
Social networks are live on the connections between people, so friend suggestions is one of the most important features of Facebook. One of the most important inputs of the algorithm for friend suggestion is most probably the current distance between you and the suggested person in the graph of friends connections. Your task is to implement efficient computation of this distance. The grader will test your algorithm against different real-world networks, such as a part of internet, a network of scientific citations or coauthorship, a social network of jazz musicians or even a social network of dolphins 🙂 You need to compute the distance between two nodes in such network. We took some of the graphs from here to use in the grader, and you can play with more of them on your own computer.
Problem Description
Task. Compute the distance between several pairs of nodes in the network.
Input Format. The first line contains two integers n and m — the number of nodes and edges in the network, respectively. The nodes are numbered from 1 to n. Each of the following m lines contains three integers u, v and l describing a directed edge (u,v) of length l from the node number uto the node number v. (Note that some social networks are represented by directed graphs while some other correspond naturally to undirected graphs. For example, Twitter is a directed graph (with a directed edge (u,v) meaning that u follows v), while Facebook is an undirected graph (where an undirected edge {u,v} means that u and v are friends). In this problem, we work with directed graphs only for a simple reason. It is easy to turn an undirected graph into a directed one: just replace each undirected edge {u,v} with a pair of directed edges (u,v) and (v,u).)
The next line contains an integer q — the number of queries for computing the distance. Each of the following q lines contains two integers u and v — the numbers of the two nodes to compute the distance from u to v.
Constraints. 1 ≤ n ≤ 1 000 000; 1 ≤ m ≤ 6 000 000; 1 ≤ u,v ≤ n; 1 ≤ l ≤ 1 000; 1 ≤ q ≤ 1 000. For Python2, Python3, Ruby and Javascript, 1 ≤ m ≤ 2 000 000.
Output Format. For each query, output one integer on a separate line. If there is no path from u to v, output −1. Otherwise, output the distance from u to v.
Time Limits.
language C C++ Java Python C# Haskell JavaScript Ruby Scala
time (sec) 25 25 125 150 37.5 50 150 150 150
Memory Limit. 2048MB.
Sample 1.
Input:
21
121
4
11
22
12
21
Output:
0
0
1
-1
Explanation:

The distance from a node to itself is always 0. The distance from 1 to 2 is 1, and there is no path from 2 to 1.
Sample 2.
Input:
44
121
412
232
135
1
13
Output:
3
Explanation:

There is a direct edge from node 1 to node 3 of length 5, but there is a shorter path 1 → 2 → 3 of length 1 + 2 = 3.
Starter Files
The starter solutions for this problem read the input data from the standard input, pass it to a blank procedure, and then write the result to the standard output. You are supposed to implement your algorithm in this blank procedure if you are using C++, Java, or Python3. For other programming languages, you need to implement a solution from scratch. Filename: friend_suggestion
What To Do
Implement the Bidirectional Dijkstra algorithm from the lectures.
Need Help?
Ask a question or see the questions asked by other learners at this forum thread.
2 Problem: Compute Distance Faster Using Coordinates
Problem Introduction
In this task you will be given a description of a real-world road network with not just edges and their lengths, but also with the coordinates of the nodes. Your task is still to find the distance between some pairs of nodes, but you will need to use the additional information about coordinates to speedup your search.
Problem Description
Task. Compute the distance between several pairs of nodes in the network.
Input Format. The first line contains two integers n and m — the number of nodes and edges in the network, respectively. The nodes are numbered from 1 to n. Each of the following n lines contains the coordinates x and y of the corresponding node. Each of the following m lines contains three integers u, v and l describing a directed edge (u,v) of length l from the node number u to the node number v.

It is guaranteed that l ≥√︀(x(u) − x(v))2 + (y(u) − y(v))2 where (x(u),y(u)) are the coordinates of u and (x(v),y(v)) are the coordinates of v. The next line contains an integer q — the number of queries for computing the distance. Each of the following q lines contains two integers u and v — the numbers of the two nodes to compute the distance from u to v.
Constraints. 1 ≤ n ≤ 110 000; 1 ≤ m ≤ 250 000; −109 ≤ x,y ≤ 109; 1 ≤ u,v ≤ n; 0 ≤ l ≤ 100 000; 1 ≤ q ≤ 10 000. For Python2, Python3, Ruby and Javascript, 1 ≤ n ≤ 11 000, 1 ≤ m ≤ 30 000.
Output Format. For each query, output one integer. If there is no path from u to v, output −1. Otherwise, output the distance from u to v.
Time Limits.
language C C++ Java Python C# Haskell JavaScript Ruby Scala
time (sec) 50 50 100 50 75 100 50 50 100
Memory Limit. 2048MB.
Sample 1.
Input:
21
00
01
121
4
11
22
12
21
Output:
0
0
1
-1
Explanation:

0 1 2
The distance from a node to itself is always 0. The distance from 1 to 2 is 1, and there is no path from 2 to 1.
Sample 2.
Input:
44
00
01
21
20
121
412
232
136
1
13
Output:
3
Explanation:

0 1 2
There is a direct edge from node 1 to node 3 of length 6, but there is a shorter path 1 → 2 → 3 of length 1 + 2 = 3.
Starter Files
The starter solutions for this problem read the input data from the standard input, pass it to a blank procedure, and then write the result to the standard output. You are supposed to implement your algorithm in this blank procedure if you are using C++, Java, or Python3. For other programming languages, you need to implement a solution from scratch. Filename: dist_with_coords
What To Do
Implement the A* algorithm from the lectures.
Need Help?
Ask a question or see the questions asked by other learners at this forum thread.
3 Problem: Compute Distance with Preprocessing
Problem Introduction
In this task you will be first given a graph of a real road network, and you can preprocess it as you wish under the preprocessing time limit. Then you will get a set of queries for computing distance, and you will need to answer all of them under the separate time limit for queries. You will have to respond to queries much faster than in the previous problem.
Problem Description
Task. Compute the distance between several pairs of nodes in the network.
Input Format. You will be given the input for this problem in two parts. The first part contains the description of a road network, the second part contains the queries. You have a separate time limit for preprocessing the graph. Under this time limit, you need to read the graph and preprocess it. After you’ve preprocessed the graph, you need to output the string “Ready” (without quotes) and flush the output buffer (the starter files for C++, Java and Python3 do that for you; if you use another language, you will have to find out how to do this). Only after you output the string “Ready” you will be given the queries. You will have a time limit for the querying part, and under this time limit you will need to input all the queries and output the results for each of the quires.
The first line of the road network description contains two integers n and m — the number of nodes and edges in the network, respectively. The nodes are numbered from 1 to n. Each of the following m lines contains three integers u, v and l describing a directed edge (u,v) of length l from the node number u to the node number v.
The first line of the queries description contains an integer q — the number of queries for computing the distance. Each of the following q lines contains two integers u and v — the numbers of the two nodes to compute the distance from u to v.
Constraints. 1 ≤ n ≤ 110 000; 1 ≤ m ≤ 250 000; 1 ≤ u,v ≤ n; 1 ≤ l ≤ 200 000; 1 ≤ q ≤ 10 000. It is guaranteed that the correct distances are less than 1 000 000 000. For Python2, Python3, Ruby and Javascript, 1 ≤ n ≤ 11 000, 1 ≤ m ≤ 25 000, 1 ≤ q ≤ 1 000.
Output Format. After you’ve read the description of the road network and done your preprocessing, output one string “Ready” (without quotes) on a separate line and flush the output buffer. Then read the queries, and for each query, output one integer on a separate line. If there is no path from u to v, output −1. Otherwise, output the distance from u to v.
Time Limits.
language C C++ Java Python C# Haskell JavaScript Ruby Scala
preprocessing (sec) 10 10 45 50 15 20 50 50 90
query time (sec) 2 2 9 10 3 4 10 10 18
Memory Limit. 2048MB.
Sample 1.
Input:
21
121
4
11
22
12
21
Output:
Ready
0
0
1
-1
Explanation:

The distance from a node to itself is always 0. The distance from 1 to 2 is 1, and there is no path from 2 to 1.
Sample 2.
Input:
44
121
412
232
135
1
13
Output:
Ready
3
Explanation:

There is a direct edge from node 1 to node 3 of length 5, but there is a shorter path 1 → 2 → 3 of length 1 + 2 = 3.
Starter Files
The starter solutions for this problem read the input data from the standard input, pass it to a blank procedure, and then write the result to the standard output. You are supposed to implement your algorithm in this blank procedure if you are using C++, Java, or Python3. For other programming languages, you need to implement a solution from scratch. Filename: dist_preprocess_small
What To Do
Implement the Contraction Hierarchies algorithm from the lectures. It is not necessary to implement all the heuristics to pass, only some of them.
Need Help?
Ask a question or see the questions asked by other learners at this forum thread.
4 Advanced Problem: Compute Distance with Preprocessing on Larger Road Networks
We strongly recommend you start solving advanced problems only when you are done with the basic problems (for some advanced problems, algorithms are not covered in the video lectures and require additional ideas to be solved; for some other advanced problems, algorithms are covered in the lectures, but implementing them is a more challenging task than for other problems).
Problem Introduction
This problem is the same as the previous one, but you will have to solve larger road networks under the same time limits.
Problem Description
Task. Compute the distance between several pairs of nodes in the network.
Input Format. See the input format for the previous problem.
Constraints. 1 ≤ n ≤ 500 000; 1 ≤ m ≤ 1 100 000; 1 ≤ u,v ≤ n; 1 ≤ l ≤ 200 000; 1 ≤ q ≤ 10 000. It is guaranteed that the correct distances are less than 1 000 000 000. For Python2, Python3, Ruby and Javascript, 1 ≤ n ≤ 11 000, 1 ≤ m ≤ 25 000, 1 ≤ q ≤ 1 000.
Output Format. See the output format for the previous problem.
Time Limits.
language C C++ Java Python C# Haskell JavaScript Ruby Scala
preprocessing (sec) 55 55 220 55 82.5 110 55 55 220
query time (sec) 5 5 20 5 7.5 10 5 5 20
Memory Limit. 2048MB.
See the examples in the previous problem.
Starter Files
The starter solutions for this problem read the input data from the standard input, pass it to a blank procedure, and then write the result to the standard output. You are supposed to implement your algorithm in this blank procedure if you are using C++, Java, or Python3. For other programming languages, you need to implement a solution from scratch. Filename: dist_preprocess_large
What To Do
Implement the Contraction Hierarchies algorithm from the lectures. It will be necessary to implement more of the heuristics and tricks than for the previous problem. You can also come up with your own ideas and techniques for speeding up the solution. We encourage you to compete on the forums, whose solution is the fastest (both in terms of preprocessing time and query time) and uses less memory!
Need Help?
Ask a question or see the questions asked by other learners at this forum thread.
5 Advanced Problem: Travelling Salesman Problem
We strongly recommend you start solving advanced problems only when you are done with the basic problems (for some advanced problems, algorithms are not covered in the video lectures and require additional ideas to be solved; for some other advanced problems, algorithms are covered in the lectures, but implementing them is a more challenging task than for other problems).
Problem Introduction
In this task you will solve the classical logistics problem called Travelling Salesman Problem: you are given the location of a depot and the location of a list of stores on a road network, and you need to find the shortest path for a truck to start in the depot, visit each of the stores to deliver the goods there, and return back to the depot.
Problem Description
Task. Compute the length of the shortest path starting in the depot, visiting each store at least once and returning to the depot.
Input Format. You will be given the input for this problem in two parts. The first part contains the description of a road network, the second part contains the queries. You have a separate time limit for preprocessing the graph. Under this time limit, you need to read the graph and preprocess it. After you’ve preprocessed the graph, you need to output the string “Ready” (without quotes) and flush the output buffer (the starter files for C++, Java and Python3 do that for you; if you use another language, you will have to find out how to do this). Only after you output the string “Ready” you will be given the queries. You will have a time limit for the querying part, and under this time limit you will need to input all the queries and output the results for each of the quires.
The first line of the road network description contains two integers n and m — the number of nodes and edges in the network, respectively. The nodes are numbered from 1 to n. Each of the following m lines contains three integers u, v and l describing a directed edge (u,v) of length l from the node number u to the node number v.
The first line of the queries description contains an integer q — the number of queries for computing the distance. Each of the following q lines starts with the integer k — the number of points the truck must visit, including all the stores and the depot. There are k more integers on the same line. The first of them is the number of the node corresponding to the depot location. The next k−1 integers are the numbers of the nodes corresponding to the store locations.
Constraints. 1 ≤ n ≤ 110 000; 1 ≤ m ≤ 250 000; 1 ≤ u,v ≤ n; 1 ≤ l ≤ 100 000; 1 ≤ q ≤ 100; 1 ≤ k ≤ 20. It is guaranteed that all the answers are less than 1 000 000 000. For Python2, Python3, Ruby and Javascript, 1 ≤ n ≤ 11 000, 1 ≤ m ≤ 25 000, 1 ≤ k ≤ 20.
Output Format. After you’ve read the description of the road network and done your preprocessing, output one string “Ready” (without quotes) on a separate line and flush the output buffer. Then read the queries, and for each query, output one integer on a separate line. If there is no path starting in depot, visiting each store at least once and returning to the depot, output −1. Otherwise, output the length of the shortest path starting at a depot, visiting each store at least once and returning to the depot.
Time Limits.
language C C++ Java Python C# Haskell JavaScript Ruby Scala
preprocessing (sec) 20 20 50 50 30 40 50 50 50
query time (sec) 16 16 40 40 24 32 40 40 40
Memory Limit. 2048MB.
Sample 1.
Input:
45
121
231
341
411
211
3
212
213
41234
Output:
Ready
2
4
4
Explanation:

For the first query, we need to start in the node 1, visit node 2 and return to node 1. The shortest path for that is to get directly from 1 to 2 and then directly back from 2 to 1. The length is 2.
For the second query, we need to start in the node 1, visit node 3 and return to node 1. A shortest path for that is to get from 1 to 2, then from 2 to 3, then return from 3 to 2, then return from 2 to 1. Another shortest path would be to go from 1 to 2, then from 2 to 3, then from 3 to 4, then from 4 to 1. The length is 4.
For the third query, we need to start in the node 1, visit all the other nodes and return to node 1. The shortest path for that is to go from 1 to 2, then from 2 to 3, then from 3 to 4, then from 4 to 1. The length is 4.
Starter Files
The starter solutions for this problem read the input data from the standard input, pass it to a blank procedure, and then write the result to the standard output. You are supposed to implement your algorithm in this blank procedure if you are using C++, Java, or Python3. For other programming languages, you need to implement a solution from scratch. Filename: travelling_salesman_problem
What To Do
First you need to compute the pairwise distances between all the stores and between the depot and all the stores, to and from. To do that quickly, you will need to use your solution to the previous problem (preprocess the graph using Contraction Hierarchies preprocessing, then find the distance between each pair of interesting locations using Contraction Hierarchies querying). After that, you have the following problem: given a graph with k ≤ 20 nodes and weighted edges, find the shortest path starting in some node, visiting all the nodes at least once and returning to the initial node. This is the classical Travelling Salesman Problem which can be solved for small values of k using dynamic programming. You can try to come up with this dynamic programming yourself, or watch this lecture about it in the next course of our Specialization, Advanced Algorithms and Complexity.
Need Help?
Ask a question or see the questions asked by other learners at this forum thread.
6 General Instructions and Recommendations on Solving Algorithmic Problems
Your main goal in an algorithmic problem is to implement a program that solves a given computational problem in just few seconds even on massive datasets. Your program should read a dataset from the standard input and write an answer to the standard output.
Below we provide general instructions and recommendations on solving such problems. Before reading them, go through readings and screencasts in the first module that show a step by step process of solving two algorithmic problems: link.
6.1 Reading the Problem Statement
You start by reading the problem statement that contains the description of a particular computational task as well as time and memory limits your solution should fit in, and one or two sample tests. In some problems your goal is just to implement carefully an algorithm covered in the lectures, while in some other problems you first need to come up with an algorithm yourself.
6.2 Designing an Algorithm
If your goal is to design an algorithm yourself, one of the things it is important to realize is the expected running time of your algorithm. Usually, you can guess it from the problem statement (specifically, from the subsection called constraints) as follows. Modern computers perform roughly 108–109 operations per second. So, if the maximum size of a dataset in the problem description is n = 105, then most probably an algorithm with quadratic running time is not going to fit into time limit (since for n = 105, n2 = 1010) while a solution with running time O(nlogn) will fit. However, an O(n2) solution will fit if n is up to 103 = 1000, and if n is at most 100, even O(n3) solutions will fit. In some cases, the problem is so hard that we do not know a polynomial solution. But for n up to 18, a solution with O(2nn2) running time will probably fit into the time limit.
To design an algorithm with the expected running time, you will of course need to use the ideas covered in the lectures. Also, make sure to carefully go through sample tests in the problem description.
6.3 Implementing Your Algorithm
When you have an algorithm in mind, you start implementing it. Currently, you can use the following programming languages to implement a solution to a problem: C, C++, C#, Haskell, Java, JavaScript, Python2, Python3, Ruby, Scala. For all problems, we will be providing starter solutions for C++, Java, and Python3. If you are going to use one of these programming languages, use these starter files. For other programming languages, you need to implement a solution from scratch.
6.4 Compiling Your Program
For solving programming assignments, you can use any of the following programming languages: C, C++, C#, Haskell, Java, JavaScript, Python2, Python3, Ruby, and Scala. However, we will only be providing starter solution files for C++, Java, and Python3. The programming language of your submission is detected automatically, based on the extension of your submission.
We have reference solutions in C++, Java and Python3 which solve the problem correctly under the given restrictions, and in most cases spend at most 1/3 of the time limit and at most 1/2 of the memory limit. You can also use other languages, and we’ve estimated the time limit multipliers for them, however, we have no guarantee that a correct solution for a particular problem running under the given time and memory constraints exists in any of those other languages.
∙ C (gcc 5.2.1). File extensions: .c. Flags:
gcc -pipe -O2 -std=c11 <filename> -lm
∙ C++ (g++ 5.2.1). File extensions: .cc, .cpp. Flags:
g++ -pipe -O2 -std=c++14 <filename> -lm
∙ C# (mono 3.2.8). File extensions: .cs. Flags:
mcs
∙ Haskell (ghc 7.8.4). File extensions: .hs. Flags:
ghc -O2
∙ Java (Open JDK 8). File extensions: .java. Flags:
javac -encoding UTF-8 java -Xmx1024m
∙ JavaScript (Node v6.3.0). File extensions: .js. Flags:
nodejs
∙ Python 2 (CPython 2.7). File extensions: .py2 or .py (a file ending in .py needs to have a first line which is a comment containing “python2”). No flags:
python2
∙ Python 3 (CPython 3.4). File extensions: .py3 or .py (a file ending in .py needs to have a first line which is a comment containing “python3”). No flags:
python3
∙ Ruby (Ruby 2.1.5). File extensions: .rb.
ruby
∙ Scala (Scala 2.11.6). File extensions: .scala.
scalac
6.5 Testing Your Program
When your program is ready, you start testing it. It makes sense to start with small datasets (for example, sample tests provided in the problem description). Ensure that your program produces a correct result.
You then proceed to checking how long does it take your program to process a massive dataset. For this, it makes sense to implement your algorithm as a function like solve(dataset) and then implement an additional procedure generate() that produces a large dataset. For example, if an input to a problem is a sequence of integers of length 1 ≤ n ≤ 105, then generate a sequence of length exactly 105, pass it to your solve() function, and ensure that the program outputs the result quickly.
Also, check the boundary values. Ensure that your program processes correctly sequences of size n = 1,2,105. If a sequence of integers from 0 to, say, 106 is given as an input, check how your program behaves when it is given a sequence 0,0,…,0 or a sequence 106,106,…,106. Check also on randomly generated data. For each such test check that you program produces a correct result (or at least a reasonably looking result).
In the end, we encourage you to stress test your program to make sure it passes in the system at the first attempt. See the readings and screencasts from the first week to learn about testing and stress testing: link.
6.6 Submitting Your Program to the Grading System
When you are done with testing, you submit your program to the grading system. For this, you go the submission page, create a new submission, and upload a file with your program. The grading system then compiles your program (detecting the programming language based on your file extension, see Subsection 6.4) and runs it on a set of carefully constructed tests to check that your program always outputs a correct result and that it always fits into the given time and memory limits. The grading usually takes no more than a minute, but in rare cases when the servers are overloaded it might take longer. Please be patient. You can safely leave the page when your solution is uploaded.
6.7 Debugging and Stress Testing Your Program
If your program failed, you will need to debug it. Most probably, you didn’t follow some of our suggestions from the section 6.5. See the readings and screencasts from the first week to learn about debugging your program: link.
You are almost guaranteed to find a bug in your program using stress testing, because the way these programming assignments and tests for them are prepared follows the same process: small manual tests, tests for edge cases, tests for large numbers and integer overflow, big tests for time limit and memory limit checking, random test generation. Also, implementation of wrong solutions which we expect to see and stress testing against them to add tests specifically against those wrong solutions.
Go ahead, and we hope you pass the assignment soon!
7 Frequently Asked Questions
7.1 I submit the program, but nothing happens. Why?
You need to create submission and upload the file with your solution in one of the programming languages C, C++, Java, or Python (see Subsections 6.3 and 6.4). Make sure that after uploading the file with your solution you press on the blue “Submit” button in the bottom. After that, the grading starts, and the submission being graded is enclosed in an orange rectangle. After the testing is finished, the rectangle disappears, and the results of the testing of all problems is shown to you.
7.2 I submit the solution only for one problem, but all the problems in the assignment are graded. Why?
Each time you submit any solution, the last uploaded solution for each problem is tested. Don’t worry: this doesn’t affect your score even if the submissions for the other problems are wrong. As soon as you pass the sufficient number of problems in the assignment (see in the pdf with instructions), you pass the assignment. After that, you can improve your result if you successfully pass more problems from the assignment. We recommend working on one problem at a time, checking whether your solution for any given problem passes in the system as soon as you are confident in it. However, it is better to test it first, please refer to the reading about stress testing: link.
7.3 What are the possible grading outcomes, and how to read them?
Good job! Hurrah! Your solution passed, and you get a point!
Wrong answer. Your solution has output incorrect answer for some test case. If it is a sample test case from the problem statement, or if you are solving Programming Assignment 1, you will also see the input data, the output of your program and the correct answer. Otherwise, you won’t know the input, the output, and the correct answer. Check that you consider all the cases correctly, avoid integer overflow, output the required white space, output the floating point numbers with the required precision, don’t output anything in addition to what you are asked to output in the output specification of the problem statement. See this reading on testing: link.
Time limit exceeded. Your solution worked longer than the allowed time limit for some test case. If it is a sample test case from the problem statement, or if you are solving Programming Assignment 1, you will also see the input data and the correct answer. Otherwise, you won’t know the input and the correct answer. Check again that your algorithm has good enough running time estimate. Test your program locally on the test of maximum size allowed by the problem statement and see how long it works. Check that your program doesn’t wait for some input from the user which makes it to wait forever. See this reading on testing: link.
Memory limit exceeded. Your solution used more than the allowed memory limit for some test case. If it is a sample test case from the problem statement, or if you are solving Programming Assignment 1,
you will also see the input data and the correct answer. Otherwise, you won’t know the input and the correct answer. Estimate the amount of memory that your program is going to use in the worst case and check that it is less than the memory limit. Check that you don’t create too large arrays or data structures. Check that you don’t create large arrays or lists or vectors consisting of empty arrays or empty strings, since those in some cases still eat up memory. Test your program locally on the test of maximum size allowed by the problem statement and look at its memory consumption in the system.
Cannot check answer. Perhaps output format is wrong. This happens when you output something completely different than expected. For example, you are required to output word “Yes” or “No”, but you output number 1 or 0, or vice versa. Or your program has empty output. Or your program outputs not only the correct answer, but also some additional information (this is not allowed, so please follow exactly the output format specified in the problem statement). Maybe your program doesn’t output anything, because it crashes.
Unknown signal 6 (or 7, or 8, or 11, or some other). This happens when your program crashes. It can be because of division by zero, accessing memory outside of the array bounds, using uninitialized variables, too deep recursion that triggers stack overflow, sorting with contradictory comparator, removing elements from an empty data structure, trying to allocate too much memory, and many other reasons. Look at your code and think about all those possibilities. Make sure that you use the same compilers and the same compiler options as we do. Try different testing techniques from this reading: link.
Internal error: exception… Most probably, you submitted a compiled program instead of a source code.
Grading failed. Something very wrong happened with the system. Contact Coursera for help or write in the forums to let us know.
7.4 How to understand why my program fails and to fix it?
If your program works incorrectly, it gets a feedback from the grader. For the Programming Assignment 1, when your solution fails, you will see the input data, the correct answer and the output of your program in case it didn’t crash, finished under the time limit and memory limit constraints. If the program crashed, worked too long or used too much memory, the system stops it, so you won’t see the output of your program or will see just part of the whole output. We show you all this information so that you get used to the algorithmic problems in general and get some experience debugging your programs while knowing exactly on which tests they fail.
However, in the following Programming Assignments throughout the Specialization you will only get so much information for the test cases from the problem statement. For the next tests you will only get the result: passed, time limit exceeded, memory limit exceeded, wrong answer, wrong output format or some form of crash. We hide the test cases, because it is crucial for you to learn to test and fix your program even without knowing exactly the test on which it fails. In the real life, often there will be no or only partial information about the failure of your program or service. You will need to find the failing test case yourself. Stress testing is one powerful technique that allows you to do that. You should apply it after using the other testing techniques covered in this reading.
7.5 Why do you hide the test on which my program fails?
Often beginner programmers think by default that their programs work. Experienced programmers know, however, that their programs almost never work initially. Everyone who wants to become a better programmer needs to go through this realization.
When you are sure that your program works by default, you just throw a few random test cases against it, and if the answers look reasonable, you consider your work done. However, mostly this is not enough. To make one’s programs work, one must test them really well. Sometimes, the programs still don’t work although you tried really hard to test them, and you need to be both skilled and creative to fix your bugs. Solutions to algorithmic problems are one of the hardest to implement correctly. That’s why in this Specialization you will gain this important experience which will be invaluable in the future when you write programs which you really need to get right.
It is crucial for you to learn to test and fix your programs yourself. In the real life, often there will be no or only partial information about the failure of your program or service. Still, you will have to reproduce the failure to fix it (or just guess what it is, but that’s rare, and you will still need to reproduce the failure to make sure you have really fixed it). When you solve algorithmic problems, it is very frequent to make subtle mistakes. That’s why you should apply the testing techniques described in this reading to find the failing test case and fix your program.
(link).
7.7 My implementation always fails in the grader, though I already tested and stress tested it a lot. Would not it be better if you give me a solution to this problem or at least the test cases that you use? I will then be able to fix my code and will learn how to avoid making mistakes. Otherwise, I do not feel that I learn anything from solving this problem. I am just stuck.
First of all, you always learn from your mistakes.
The process of trying to invent new test cases that might fail your program and proving them wrong is often enlightening. This thinking about the invariants which you expect your loops, ifs, etc. to keep and proving them wrong (or right) makes you understand what happens inside your program and in the general algorithm you’re studying much more.
Also, it is important to be able to find a bug in your implementation without knowing a test case and without having a reference solution. Assume that you designed an application and an annoyed user reports that it crashed. Most probably, the user will not tell you the exact sequence of operations that led to a crash. Moreover, there will be no reference application. Hence, once again, it is important to be able to locate a bug in your implementation yourself, without a magic oracle giving you either a test case that your program fails or a reference solution. We encourage you to use programming assignments in this class as a way of practicing this important skill.