Blog, ECS 145 Winter 2020

Please note that my handin username is "matloff". See syllabus.

I just sent out the Group Quiz grades. If you didn't receive yours, CONTACT ME IMMEDIATELY. It is highly important.

To ECS 145 and 189G students:

In submitting your Term Projects, make sure -- ABSOLUTELY, TOTALLY, POSITIVELY, EXTREMELY SURE -- that you get the file name right, with proper e-mail addresses etc. In grading the ECS 145 group quiz just now, two of the submissions were wrong. One used commas instead of periods and the other had only the submitter's name, not the other three team members. Fortunately I caught it in time, and changed the file names by hand, but potentially SEVEN students could have had an F grade on this quiz.

As you have probably heard, all in-person final exams are canceled. But since our class didn't have a final anyway, there is no effect on us.

We have a new supplement. Make sure you understand this and the other supplements well to prepare for Friday's Group Quiz.

I was not happy with various parts of the supplement on R classes and environments, so I have revised (and slightly expanded) it.

Download and print the new supplement on R classes and environments Consider it an official part of our course materials, and bring it to lecture tomorrow.

We will not have lecture next Wednesday. I have a medical appointment that day, made a couple of months ago, difficult to schedule. Sorry for my absence.

Tomorrow's quiz is canceled. Next week's quiz will cover only Python, not R.

There likely will be a question on SimPy in tomorrow's quiz. Please make sure that SimPy/ is in your OMSI directory.

Our Term Project is ready!

We'll move to R now, starting with tomorrow's lecture. I've spent more time on Python than I had planned, but that is no problem as far as I'm concerned. My goals, as stated at the start of the quarter, have been to teach you the issues involving scripting languages, and of "practical" issues in software development in general. The actual language we've used is less important. This is not a "Python course" or an "R course."

Nevertheless, you will indeed have had plenty of experience with R by the end of the quarter. You will have 7 lectures, 2 quizzes and a term project, all in R.

Our quiz this Friday will cover material through yesterday's lecture.

% python Elevator.py 1.0 5 2.0 25000
mean passenger wait:  2.49207188575
prop. of visits that leave passengers behind:  0.147056203794

Partial tests for Hwk II.A:

% python Elevator.py 1.0 5 1.5 25000
mean passenger wait:  1.65713234398
% python Elevator.py 1.0 10 1.5 25000
mean passenger wait:  1.50607292206
% python Elevator.py 1.0 20 1.5 25000
mean passenger wait:  1.49798548738
% python Elevator.py 1.0 20 2.0 25000
mean passenger wait:  1.99757145113
% python Elevator.py 1.0 5 2.0 25000
mean passenger wait:  2.49207188575

We will be starting R soon, either Friday or Monday. Please read my R tutorial for non-techies in preparation.

There will be no quiz or discussion section this Friday.

I was asked what should be done with the leftover waiting passengers at the end of the simulation. The answer is, Nothing! Remember, because maxSimTime is finite, the answers are only approximate, so the larger that quantity is, the better. If it is larger, the impact of the leftover passengers is tiny.

I've significantly changed -- i.e. weakened -- the requirements for Hwk II.A. It's still plenty hard!

I'll post some test cases as soon as Sarmishta finished verifying my solution.

Any kind of parallel programming is REALLY HARD. One must deal with multiple simultaneous events, which is quite difficult, BOTH in the writing AND in the debugging stages. The timing issues -- what happens before what -- can be extremely delicate.

DES programming, Part A of Hwk II, is no exception. I urge you to get started as soon as possible. I will be posting some test cases and some suggestions as to approaches to writing the code.

Part A of Homework II is now on the Web!

Remember, all quizzes are cumulative. Quiz 4 will tentatively have a question involving networks, and one using the Python zip() function.

Notice posted at the request of the SDC:

The Student Disability Center (SDC) asks that students interested in serving as paid note takers for this course [ECS 145] please contact the Student Disability Center at sdc@ucdavis.edu. Note takers are paid a stipend of $25 per unit*, but must have a social security number or have visa status to qualify for an SSN, have a GPA of 2.5 o r better, and be in good Academic standing with UC Davis. Students should attach a sample of their notes to the email and put the following information in the subject line: 'note taker', the course code, CRN number, and instructor's name. If selected, the SDC will offer the position to the Note Taker. Notes provided to SDC students needing note takers are not to be posted, sold, or otherwise distributed, either by the note taker or the SDC Student receiving the notes. Note takers and SDC students receiving notes are expected to comply with the UC Davis academic honor code. Thank you in advance for your time and consideration * You can volunteer in lieu of payment

A sharp student asked me about the code in Sec. 6.1.4.1. Several points here.

First, two changes must be made. In x.py, change line 12 to

i = raw_input() + '\n'

Here is why: As you know, makefile() at the client here will repeatedly input bytes from the server, forming lines according to the EOL (end of line) characters it receives. If for instance the server sends 'abc\nde22', the first line, 'abc\n' will now be ready the the iterator flo in line 11 will set l to that line. Meanwhile, makefile() will save the partial line it has, 'de22', for the further bytes that will come for the second line.

The problem with the code as shown in the book is that raw_input() does not add its own EOL character. Without that makefile() at the client will wait forever for the first line!

On the other hand, line 12 in the client must be changed to

print l[:-1]

to cut OFF that EOL character, because print will add one on its own.

The student who brought this example up with me wondered why there is no s.recv() call in the client. The reason is that makefile() is repeatedly calling that function internally.

Due to a meeting over in the Genomics Center, I'll be leaving my office hour at 4 pm this Friday.

Today two students came to see me regarding Question 3 in Quiz 2. They stated their concerns very thoughtfully, and I am pleased that they felt that they could approach me on this. Everyone, please feel free to bring anything up. But I have something important to say here.

The two students' solutions were wrong because their midReadlines() function read the entire file, rather than reading starting at the designated line, as the problem had specified. But they felt that since their output was correct, they should get full credit for the problem. But the output only checks part of the problem. (Note that that was also true for Question 4, Quiz 1.) Any exam problem measures a student's insight into one or more concepts of the course, and Question 3, Quiz 2 was measuring students' insight into the issue of Python execution speed.

• Our emphasis from Day 1 has been that "This course is not about language syntax. (No course should be.) You'll learn the syntax of course, but our focus will be on the reasons behind the language design, what the goals of the language are, how to deal with the slowness, etc."
• We've discussed a number of times the intepreted nature of Python and R -- what it means, the need to compensate for slowness etc.
• This was especially an issue in our coverage of Python file operations. I drew a diagram of disk structure, and its implication for file access speed -- seek time, rotation access time, disk defragmentation and so on.
• It was noted on several occasions that things in the real world are often of huge scale, e.g. datasets with hundreds of millions of lines at Google, the Bank of America's crucial dependence on optimizing the placement of various files on disk.
• One available tool is the seek() function, which allows one to directly go to specific byte in the file, without starting at the beginning -- thereby potentially saving huge amounts of read time.

This then was what Question 3 was all about. By forming an index of byte positions of the lines of the file, we can go directly to any line, without starting at the beginning of the file. This is what the phrasing of the problem says:

...midReadLines(fname,fnameIdx,startLine) will read the file fname, starting at line number startLine.

Question 3, as a tool for measuring student insight, was designed to assess how well a student can go beyond the syntax of Python, to deal with the slow speed of the language, in this case speeding up file access.

And in doing that assessment, the prompt did not say, "Be sure not to start reading at the beginning of the file." A student who understands the Big Picture would not need such phrasing; the need to start at the designated byte position would be obvious to him/her. And indeed it was obvious to a number of you; I think 10-12 of you got this right.

'Nuff said, see you Wednesday.

I just sent out your Quiz 2 grades. The solutions are on our class Web page.

Pay close attention to Question 3. If in midReadLines() you read in the entire file, you are defeating the purpose of having an index.

For Quiz 2, you MUST have in your OMSI directory the files I've placed here. So for instance the files x and y, and the directory w, must be in the same directory as OmsiGui.py.

Recall the blog post of Tuesday, January 21, 4:00 pm. Make sure you still have the files x and y for Quiz 2.

BTW, did you notice a connection in the solution to Question 4 on the quiz? Look at the line

      textfile.ntfiles[fname] = self  

Only three students got this right. That's OK, as the last problem in a quiz is intended to be the hardest, and these three will get Extra Credit. But...

Actually, this was assessing your insight into our section on linked data structures in Python. (Binary tree example.) Remember, a key "big picture" issue there was how to make links without pointers. Python does it by assigning by the name of the object to be pointed to. E.g. 'a = b' points 'a' to 'b', and in the tree case 'b' would be a subtree etc.

The questions and answers for Quiz 1 are now on our Web site.

If you believe you were misgraded on a quiz problem, please feel free to bring this up with me. Make sure to read the solution first, and send me an e-mail query.

I was asked whether tda() is supposed to print anything. Definitely not! Do sqrt() and sort() print anything?

I've clarified the description of the input file in the Homework specs, and now specified the name for your code file.

REMDINDER: Your code file must consist of functions only, no free-standing code.

On quizzes, in order to get full credit on a problem, your code must actually run. Be sure to run it in OMSI!

In the Homework specs, "the produced list" means "the returned list." In the scenario given, the function would return a single list (not a list of lists etc.) of 57 numbers.

Some of you may be concerned that we are using "only" Python 2.7. It is now deprecated, starting January 1, but (a) it will remain as a major Python version for a long time (e.g. on CSIF) and (b) the differences really are minor. Our ace developer Matt Donnelly has converted OMSI to Python 3. We'll probably switch to it later in the quarter, but take a look at his git pull; the differences are mainly parentheses in print statements and change in network actions from character strings to byte strings.

I just finished adding language to clarify the Homework problem. I've also slightly reduced the scope.

Concerning Quiz 1:

• Please create two text files in your OMSI directory, x and y, with contents

  abc
  de
  f
  

  and

  168
  88
  

  respectively. They will be used in the quiz.

• The best way to prepare for the quiz, other than reading the book thoughtfully and pondering the class discussions, is to work on Homework I as much as possible by the time the quiz rolls around.

Problem A of Homework I is now on our Web page. I originally planned to have a Problem B, but I will just leave it this way.

Our syllabus is ready.

The purpose of Quiz 0 is to show that you are ready to use OMSI, in BOTH Python and R. You are NOT expected to know R yet, but you are expected to be able to run, say, print(1+1) in R.

Most important, you MUST have R installed properly on your machine, and the proper search path configured, as stated in the OMSI docs.

ECS 36A and ECS 34 include Unix tools in their topic coverage. Thus you should be very familiar with the concept of a search path. If not, you are not prepared for this course, and will need to learn these things quickly.

If you haven't run OMSI in both instructor and student roles as the docs state, with both Python and R problems, then you are not ready for Quiz 0.

On quizzes, you are restricted to using only Python constructs that we've covered so far in the book. On programming assignments, basically the same rule, but I'll entertain rquests for exceptions.

Dynamic creation of instance variables, use of __dict__:

# (code to be placed after line 23, p.26

a.u = 3  # legal!
textfile.u = 3  # legal too!
a.u  # prints 3
b.u  # error
textfile.u  # prints 3
a.__dict__  # prints instance variables, incl.u
b.__dict__  # prints instance variables, no u
textfile.__dict__  # some surprises

If you added the class after the first lecture, make sure you go to our class info page., and its links to our blog and the pre-quarter heads-up.

As I mentioned earlier, our course syllabus is not quite ready. Should be ready in a couple of days, and I'll announce it here.

Our office hours are now listed in our class info Web page.

Note again that you will need Python 2.7 on your laptop in order to use OMSI. Conversion of OMSI to Python 3 should be simple, but we probably will not get to it this quarter.

You'll need to be an expert on OMSI, our online exam system, before discussion section of next week. We will have Quiz 0 then, whose sole purpose is to ensure that students are ready to use OMSI in the subsequent quizzes.

This should be an automatic A+. And for most students in ECS 132 last quarter, it was:

> z <- read.table('Quiz0Grades')
> table(z[,7])

  A  A+   B   C   D  D+   F
  1 106  20   1   9   5   1

You can avoid getting any grade lower than A+ by simply (a) reading the docs carefully and (b) actually running OMSI, playing the roles of both student and instructor. In (b), make sure your experiment has both Python and R questions (could be just printing out 2+2); the R question is especially important, to make sure you don't have search path problems.

You will need both Python 2.7 and Python 3 on your laptop. As of January 1, Py 2.7 is officially deprecated. OMSI is written in Python 2.7, but according to the ECS 189G TA Runtian Wang, only a few print and exception statements need to be changed. We'll switch sometime later in the quarter. For now, it's 2.7.

Make sure to adhere to our OMSI rules.

The correct version of our Python book is http://heather.cs.ucdavis.edu/~matloff/Python/PLN/FastLanePython.pdf There was an error on the info page, my apologies.

Please note that I will be announcing various supplements for both Python and R later in the quarter.

News:

1. I've updated our class info page and fixed links in the prequarter heads-up page.

2. According to the Registrar site , our class' scheduled final exam slot is Tuesday, March 17, 1:00-3:00 p.m. We have no written final, but the Term Project deadline is 11:59 pm on that date.

As noted in class, I will assign your Term Project about three weeks prior to that deadline. As also noted, the Project IS the class, extremely important. The generous grading scheme reflects that.

3. Remember to bring your textbook to class on Wednesday.