Blog, ECS 145 Winter 2017

March 24, 9:55 p.m.

Several people have sent me e-mail this week, and I said I would reply after I finished the group quiz. So, you should hear from me by tomorrow.

March 24, 9:50 p.m.

Your cumulative quiz records, including Quiz 7, the group quiz, are being mailed to you as I write this.

March 21, 2:05 p.m.

As I've said, the term project is more flexible than the homework, so reasonable deviations are OK.

But in the specs, it says that your goal here will be similar to that of the Python binary tree code in our book. In particular, that means that your print.bintree() must print out the contents of the tree, in sorted order, just like our Python example.

March 20, 4:45 p.m.

Concerning packaging in the term project, you will submit a .tar file. When I unpack it (tar xf ...), at a miniumum I will see -- in the same directory from which I invoked the tar command -- a .tex file and associated .pdf, and the tar.gz file constructed by the build of your package.

March 19, 3:45 p.m.

I am mailing out the grades for Quiz 6, and earlier sent out the ones for Quiz 5. My apologies for the delay.

Well over half the class got a score of 100 on Quiz 6. Good job!

I will be grading the "stragglers" next, i.e. the ones who turned in on paper. Those students will learn of their grade when I send out my cumulative report after I grade the group quiz.

This week will be extremely busy. I have two group quizzes to grade (145 and 132), and various other tasks that I put off doing because of quiz composing and grading during the last week.

For that reason, please bear with me if you send me e-mail queries on your quiz results. I will definitely get to them, but it may take a couple of days. Feel free to nudge me if you don't hear back from me by Tuesday evening.

March 17, 2:10 p.m.

Concerning the term project problem with trees, etc., my formulation of the S3 version wasn't quite right, but in a way I'm glad I made this mistake. Here I will first state what you can do, and then say why this is a great learning opportunity.

The problem is that R, as a functional language, tries to be free of side effects. That's the whole point of the R Core team adding reference classes a few years ago -- to add an exception to the "no side effects" rule.

Without that we run into a problem in an S3 implementation of a tree, for instance. In doing a pop() operation, the tree will change, and thus by the "no side effects" property, need to reassign the new tree. That cannot be done directly in R for S3. There are various indirect ways, though.

My favorite approach would be to pass the name of the tree as an argument to pop(), then use assign() to reassign the new tree.

Remember, as noted before, this is not like a homework assignment where there is one uniform test function that your code is run through. Instead, I will run the examples in your man pages, for instance.

At any rate, this really shows why I don't like functional languages.

March 14, 10:10 p.m.

GROUP QUIZ ON FRIDAY:

Yet another reminder about our group quiz on Friday, during class time. Note especially the last bullet:

March 14, 7:25 p.m.

My apologies. I still owe you grades on the last two quizzes. Just overwhelmed with end-of-quarter stuff, plus a need to update the grading part of OMSI. Soon...

March 14, 8:45 a.m.

Following up on my post of 11:45 p.m. of last night:

By allowing you to use reference classes, I was implicitly allowing other changes in the specs as well. So, this is very different from a homework assignment, where the TA ran your code through a script. Instead, what I do in projects is look over your actual code directly. In terms of running your code, I have the examples you'll put in your man/ directory files.

Once again: What I am most interested is your report, in which you explain your rationale for writing your code the way you did.

March 13, 11:45 p.m.

Several people have asked me if they can use reference classes for the term project. The answer is yes, providing your report discusses the issues of S3 vs. reference classes vs. others (e.g. one called R6).

Don't underestimate the importance of your written report. It is what distinguishes the term project from a homework assignment.

March 13, 2:15 p.m.

Last week I stated that you would need to know how to do matrix multiply for today's quiz. He wondered why that turned out not to be the case.

I had originally planned to have two problems on today's quiz. In addition to the "walk" problem, today, I was going to have another problem, on finding the largest eigenvalue of a matrix. I would have defined this in the question, but the key point is that the algorithm involves taking powers of the matrix, renormalizing each time, hence my announcement. But in the end, I felt that the two problems were too much for our alloted time, and I removed the matrix one.

March 13, 2:10 p.m.

Due to going to dinner with the faculty applicant, I will not be able to hold my office hour today.

March 12, 8:00 p.m.

"Traditional" push/pop operations:

Push returns the new version of the data structure. (Note the performance penalty.) Pop returns the popped value.

March 12, 2:30 p.m.

READ NOW OR FOREVER HOLD YOUR PEACE. :-)

March 11, 2:30 p.m.

In our work with both Python and R, I've always emphasized my motto, "When in doubt, try it out." Please note that you can do that in OMSI. Just put your little test example in your OMSI answer window.

For example, if you are not sure whether the ^ sign is the right exponentiation symbol for R, just put

2^3

in the window, and save and run. You can even get to the online help, e.g. placing

help(lm)

in the window.

March 11, 7:40 a.m.

DIRECTIONS FOR OUR GROUP QUIZ, MARCH 17:

March 11, 7:20 a.m.

PLEASE NOTE: I strongly recommend that you bring a USB key to both our Monday regular quiz in discussion section and our group quiz in lecture on Friday. That way, if there is suddenly a WiFi problem, you can copy your answer files to the USB key and turn that in.

Make SURE your name is shown in a README file on the key.

For Monday's quiz, where we will again use OMSI, your answer files are in the same directory from which you invoked OMSI. The file names are of the form omsi_answer*.

March 10, 9:10 a.m.

IMPORTANT R NEWS:

March 9, 11:40 p.m.

ECS 145 news:

March 7, 8:00 p.m.

Yesterday I speculated to students my office that the odd behavior of the pixmap package was due to use of different pixel sizes (e.g. 8 bits) on different machines. An alert student seems to have confirmed this, noting the max argument. In any case, though, use of round() should be fine.

March 6, 3:50 p.m.

I have investigated the file-saving issue, and found no problem. Everything worked fine. I didn't even have to do any rounding.

It could be that some people did not use write.pnm() carefully, and didn't set the file and type arguments properly.

The materials you submit do not have to have any explicit code using write.pnm(). However, to be safe, I'd recommend saving your modified file, exiting R, then restarting R and reading in the file to make sure the message is intact.

Due date is still tomorrow, as I stated in my last message.

March 6, 12:50 p.m.

There may be an issue with the saving of files in Problem A. I will look at this later today, but in any case I am moving the due date back one day, to tomorrow.

March 5, 9:10 p.m.

In Problem A, assume .pgm files.

March 4, 5:10 p.m.

I just inserted this clarifying remark in Problem A:

...or you run out of pixels. Once you do successfully change some pixel, start from there: Try to insert the next character of the message stride pixels away from the one you just changed.

March 3, 3:25 p.m.

Note that in both Problems A and B, you are on your own in terms of finding the necessary R constructs. I would note, for instance, that Chapter 11 of our book is on files, directories and so on, and Chapter 18 is on strings. You should not have to go to the Web, but of course are welcome to do so; of course, any actual code you use from the Web must be properly cited.

March 3, 12:55 p.m.

I've been recommending that you attend certain department seminars that I believe you would understand. Here is the next one, to be held next Wednesday:

Computer Science Seminar: Zhou Yu from Carnegie Mellon University

When: Tuesday, March 7th at 3:10pm

Where: 1131 Kemper Hall

Title: Situated Intelligent Interactive Systems

Abstract:

Communication is an intricate dance, an ensemble of coordinated individual actions. Imagine a future where machines interact with us like humans, waking us up in the morning, navigating us to work, or discussing our daily schedules in a coordinated and natural manner. Current interactive systems being developed by Apple, Google, Microsoft, and Amazon attempt to reach this goal by combining a large set of single-task systems. But products like Siri, Google Now, Cortana and Echo still follow pre-specified agendas that cannot transition between tasks smoothly and track and adapt to different users naturally. My research draws on recent developments in speech and natural language processing, human-computer interaction, and machine learning to work towards the goal of developing situated intelligent interactive systems. These systems can coordinate with users to achieve effective and natural interactions. I have successfully applied the proposed concepts to various tasks, such as social conversation, job interview training and movie promotion. My team's proposal on engaging social conversation systems was selected to receive $100,000 from Amazon Inc. to compete in the Amazon Alexa Prize Challenge (https://developer.amazon.com/alexaprize).

March 3, 12:25 p.m.

In Problem A, I have fixed a typo (see the point about being relatively prime), and added a requirement that the embedded secret message be terminated with a null byte. By the way, concerning the consec argument, you won't be checking diagonals, just rows and columns.

For Problem B, it may be helpful to think in the following way: One of the nice features of Python is the os.path.walk() function. Wouldn't it be nice to have that function available to the R community? You will be writing such a function! In fact, I hereby am announcing a prize (no, not monetary :-( ): The team that comes up with the best implementation of Problem B will be praised publicly in my R blog, matloff.wordpress.com. The judge will be the TA (I will be a tiebreaker if needed), and the judging criteria will be quality and clarity of code, including a clear description with examples in the comments. In any event, get out of the "I'm doing this just because my course requires this" mentality!

March 2, 10:25 p.m.

I've gotten several queries concerning Problem A, and the consec argument.

Try to keep in mind the goal: We don't want people who view the image to notice that there is "something wrong" with it, and then possibly discover the hidden message. While a person viewing an image would not notice any "blips" in the image of the altered pixels are isolated from each other, it is different if they are all contiguous in the same row, or all contiguous in the same column.

The question is then, Where do we draw the line? How many contiguous altered pixels do we deem "too much," risking viewers thinking there is something wrong with the image? We let the user -- i.e. the person hiding the message -- indicate that cutoff point via the argument consec.

March 2, 9:05 p.m.

As stated in class yesterday, both problems in Hwk III are due Monday.

I fixed a couple of minor typos.

March 1, 10:25 p.m.

I will have a special office hour Thursday, 2:45 to 3:30.

March 1, 12:50 a.m.

Due to the schedule of our speaker tomorrow (Wed.), I will not be able to hold my usual office hour.

I do encourage to attend the talk, 3:10 in 1131 Kemper. The topic will be machine translation from Chinese to English; should be very interesting.

February 28, 5:55 p.m.

Of course, my 3:25 posting was in error, intended for ECS 132.

February 28, 3:25 p.m.

Quiz 7 will cover all material through Chapter 11, including Appendix B (and A).

February 28, 12:05 a.m.

I have improved the documentation etc. in OMSI. Please test the new version AS SOON AS POSSIBLE. Here are directions (must use Python 2.7):

Further documentation is in the README.md file.

February 27, 6:00 p.m.

Problem B in Homework III is now ready.

February 27, 3:40 p.m.

Note that R's c() function is named concatenate because that is exactly what it does, e.g.

> x <- 1:3
> y <- c(5,12,13)
> c(x,y)
[1]  1  2  3  5 12 13

February 27, 1:15 p.m.

Please note that we are using SimPy Version 2.1. That is what is installed in our class site, ~matloff/Pub/SimPy; see blog post, February 22, 8:10 pm).

February 26, 11:35 p.m.

Meanwhile, concerning Problem C, Hwk II, I realize now that the result of the test case, for a much larger run time, should be (0,1).

February 26, 11:30 p.m.

Here is more concerning why I canceled tomorrow's quiz.

There were at least two students who were still struggling to make OMSI work for them. Though I am fairly sure it was a problem in the way they were using it, the fact is that I really didn't give you enough lead time on this. The server was only announced on Thursday, and then there was trouble with keeping it up for long periods of time, which I am fairly sure now was a timeout issue.

I have used OMSI successfully in a couple of classes; it DOES work. But I do want to check it further before trying it in our class.

In addition, the documentation is very poor. The students who designed it last year, under my guidance, were all brilliant programmers, but they didn't put enough time into documentation. I am currently cleaning up the README file, and other aspects of the docs.

I will announced the revised version in a couple of days. I'll need you to test it right away, no dawdling.

February 26, 7:50 p.m.

In order to make sure things go smoothly with OMSI, I am hereby canceling our quiz tomorrow. The quiz on March 6 will cover only Python.

Thong and our Student Assiatant, Dasha will be at the discussion section anyway tomorrow, to help anyone who has been having trouble using OMSI.

February 26, 2:50 p.m.

I've now placed Problem A of Homework III on the Web. DUE ONE WEEK FROM TOMORROW! Problem B will follow soon.

This will be you last homework assignment. I will announce the term project in about 10 days.

February 26, 12:40 a.m.

Reminder: Monday's quiz will use OMSI. Bring your laptop, charged up, Internet-ready, with Python installed and in your search path.

NOTE CAREFULLY: During the quiz you are not allowed to iconify, move or shrink the OMSI window. It must be the only item showing on your screen. Other than your OMSI connection, you are not allowed any other Internet communication.

February 23, 12:20 p.m.

TREAT THE FOLLOWING AS A REQUIRED COURSE TASK.

Thong is running the OMSI server on pc29, port 5000. Test it as soon as possible. Be sure to use your official UCD e-mail address, so that Thong will have a record of your testing OMSI.

February 22, 9:40 p.m.

Please note a couple of things regarding Problem C:

February 22, 8:10 p.m.

I just fixed a typo in the test case for Problem C.

Also, please note that I installed SimPy on CSIF in ~matloff/Pub.

February 22, 3:00 p.m.

I just learned today that I had not yet placed the Quiz 4 solutions on our Web page. I have done so now.

February 19, 3:20 p.m.

I've added a little more to Problem C, mainly in providing a (partial) test case.

February 18, 2:30 p.m.

We will take our quiz on Monday, February 27 using the OMSI system. You will need to bring your laptops, fully charged, with Python installed and in your search path. If this would be burdensome to you, please discuss this with me.

It is REQUIRED that you try the system before the quiz, so that nothing goes wrong. The TA will set up an OMSI server and announce instructions for your dry run.

February 17, 11:00 p.m.

I'm mailing out your Quiz 4 grades as I write this. Overall, they were quite good!

As soon as that mailing is done, I'll mail out the cumulative records.

February 17, 12:15 a.m.

A rough draft of Problem C is now on the Web.

February 17, 12:10 a.m.

There is something I mentioned briefly in class that I should amplify on.

When we say that threads share global variables, that can be more general than ordinary globals. For instance, consider a class variable. It is shared by all the instances of the class, and hence is global in spirit in that sense. But in addition, that class variable is also visible anywhere in the program (within the same module), so it really is global. (In C++ it is actually implemented that way.)

Taking this further, any memory address within a program's address space is visible throughout the code. Say function f() launches several threads running function g(), and f() invokes g() with an argument x which is local to g(). Then x is global to all the invocations of g().

February 16, 11:40 p.m.

Not quite lucky enough -- ran out of energy. :-( Long, mysterious debugging session, finally fixed.

Advice: When writing a subclass, mysterious errors may occur if you forget to invoke the constructor of the parent class. :-(

February 16, 9:10 p.m.

With luck, I will have the specs for Problem C up some time this evening.

Quiz grades coming soon, hopefully tomorrow. I will also run another cumulative grade report, which among other things will serve to notify those who got scores of -1 what their new grades are.

February 16, 1:00 p.m.

Regarding my message yestday on the SimPy tutorial, make sure to bring the hard copy to class next Wednesday.

Also, starting next Friday, starting bringing our R textbook to class.

February 15, 11:30 p.m.

We will be using the SimPy language in Problem C. Please print a hard copy of my tutorial, which is considered part of our course materials (e.g. eligible for quizzes). I will lecture on it next Wednesday.

February 15, 9:10 p.m.

I've add a small test case to Problem B.

February 14, 4:10 p.m.

I just put Problem B on the Web. There will also be a Problem C.

February 12, 1:30 p.m.

Here is where we are headed in our course. We will finish Chapter 5 and cover Chapter 6; Problem b of your current assignment will make use of both of those chapters. Chapter 7 will not be considered part of our course. We will then switch to R. We will have one or two assignments using R (probably one assignment with two problems), and R will also be used in your term project.

February 11, 12:50 p.m.

I am mailing out the grades on Quiz 3 as I write this. There was a record number of scores of 100 (10) but also a record number of scores of -1 (also 10).

The solutions are on the Web. Please pay particular attention to Problem 2(c). We are sorting the textfile objects, not the numbers of lines.

February 10, 9:20 p.m.

Our quiz Monday will cover all pages through 93 (including all of Chapter 4).

February 10, 3:20 p.m.

I have further extended the test case for Problem A.

February 10, 11:05 a.m.

In the specs for Problem A, it says, "And if we had had, say, a (5,4,1) record in our data, that would count 2/3 as well." What that meant was that the (5,4,NA) would count as a 2/3 match to (5,4,1), not that the (5,4,1) count as a 2/3 match to (5,4,5).

I've clarified the final sentence in that paragraph to read, "Partial matches are made only of nonintact patterns to intact patterns."

February 9, 9:55 a.m.

When I give a programming assignment, I usually do not write out the code myself, but I did so in Problem A. Our TA Thong also wrote a version of it, and it is interesting to compare. The difference is rather striking, due to a major difference in style. Of course, neither his style nor mine is "better," the difference is worth commenting on.

Interestingly, the actual code he and I have is pretty similar.

Again, there is no "correct" style, and good programmers will vary widely in style. This is an interesting demonstration of that.

February 9, 9:50 a.m.

I have placed a slightly longer test case in the Homework II specs, and have set the due date for Problem A to February 16.

February 8, 9:55 p.m.

In Problem A of the homework, don't worry much about speed or space. This is a very difficult problem to optimize!

I'll have test cases up later this evening or tomorrow. Meanwhile, just use the example as the test case.

Note: When checking whether a record with NAs is a partial match, check ONLY to intact records in the data set.

February 8, 11:25 a.m.

News:

February 8, 9:05 a.m.

Tomorrow, Thursday, there will be a talk by Sam King of Lyft, titled, "Fighting Bad Guys: How to Secure Products at Scale." I urge you all to attend. Here is the abstract of the talk:

In this talk, I will describe some of the security problems we faced at Lyft and Twitter, and the techniques we used to solve them. I will also discuss previous research on designing and defending against malicious hardware.

The talk is at 3:10, 1131 Kemper.

February 7, 7:55 p.m.

I have now placed Problem B of Homework II on our Web page. It will add a test case later. Also, I will be adding Problem B.

February 7, 12:55 p.m.

Two news items:

February 6, 8:35 p.m.

I will periodically e-mail your cumulative quiz grades. For those who had -1 scores due to formatting and resubmitted, this will be the mechanism under which you will learn your new grade. Of course, you can always ask me individually too.

February 6, 3:10 p.m.

The assignment specs ask you to keep an fs file contiguous, meaning the within a file's allotted space it does not have an "island" of nulls between non-nulls. If you did not allow for this, the time is getting late, so don't worry about it.

February 5, 12:10 a.m.

The code on p.66 was not quite right. Here is the corrected version:

def dividetomax(p,m):
   k = 0
   while True:
      if m % p != 0: return (p,k)
      k += 1
      m /= p

def primefact(primes,n):
   def divmax(p): return dividetomax(p,n)
   tmp = map(divmax,primes)
   print tmp
   tmp = filter(lambda u: u[1] > 0,tmp)
   return tmp

You will find it helpful to run this code yourself, and maybe tweak it.

February 4, 9:25 p.m.

If your answer on Problem 3(a), Quiz 2 was

fname.index('.')

I might be able to add more points added to your score. Please let me know.

February 3, 9:15 a.m.

From time to time I will mail out your cumulative quiz records, which I am doing as I write this.

February 2, 4:45 p.m.

If you need to send me program output, please do not attach a screenshot. An image is difficult for me, in that I sometimes will read your mail from low-bandwidth locales, and more important, I can't capture the code and try it myself.

Instead, use the Linux/Mac script command, which records a shell session. See the man page.

February 2, 11:15 a.m.

I mentioned that we may use the OMSI system for one or more of our quizzes. This is quite advantageous to students and instructors. For the former, it means you can actually run your code and if need be, fix it, before submitting it. And we avoid the bother of having both paper and electronic submissions.

OMSI is written in Python, and was developed last year by a team of 6 students, and myself.

You will need to have a laptop with Python (and later R) installed. If you don't have one and cannot borrow one, you can submit your quiz on paper. If anyone sees any problems, please let me known.

I have in mind using OMSI on MONDAY FEBRUARY 13.

February 2, 11:05 a.m.

The solutions for Quiz 2 are now up on the Web. Let me know if you have any questions.

February 2, 8:30 a.m.

Apparently there is a homework group in which some students had done a somewhat similar assignment as ours in ECS 150. I did anticipate that a 150 class might have done something like this sometime, but I did not anticipate that a homework group would change our specs to match those of their 150 assignment.

In my blog post of Jan. 25, I gave a little example. Your code must comply with this.

As I have said many times, the TA will be writing a script to test your code. Taking liberties with the assignment specs as above will probably mean that your code fails his test.

This group made a very serious error in judgment. In industry, this could get someone fired. Of course, it's better to learn this lesson now than later. And anyone can make a mistake. So, I am extending the due date until Monday, so that any group that had written noncomplying code can fix it. Of course, other groups get the extra time too, so they can thank this renegade group. :-)

AGAIN: In all of our assignments, the TA will be writing a test script that he runs your code on. Deviations from the assignment specs will likely fail his test.

February 1, 11:50 p.m.

My script is mailing out your Quiz 2 grades as I write this. Overall I was pleased with the many high scores, though of course I'd like to see fewer low ones.

IF YOU DID NOT RECEIVE ONE OR BOTH QUIZ GRADES, NOTIFY ME IMMEDIATELY.

February 1, 9:10 p.m.

With fs.mkdir(), you do not have to allow for a case like

fs.mkdir('a/b/c')

if a and b do not already exist.

February 1, 8:10 p.m.

As mentioned before: In our assignment I only asked you to check for a few specific errors, such as trying to open an fs file when there is insufficient room for it. You are NOT being asked to check for other kinds of errors, which are extremely numerous.

This evening someone asked how to disable functions like fs.open() at a time that the fs system is suspended. That would be a great question if you were writing this as a production system in real life. But again, that is not what we are doing; it would be almost impossible for you to guard against hundreds of possible problems during the time frame of our assignment.

Note that suspension of fs really means it. The user should be able to suspend the system, exit Python and log off the machine, and then come back the next day, start Python and resume fs.

February 1, 6:00 p.m.

Two important notes on the current assignment:

January 30, 12:55 p.m.

I've made some clarifications to the assignment specs. I've made it explicit that you must have an fs.chdir() function; I've explicitly added an argument dirname to fs.dir*(); and made it explicit that the sole purpose of the native file is to provide space for your private file system; the contents of the native file are overwritten.

January 29, 9:15 p.m.

Our TA has kindly set up a Piazza site for the class.

I agree that Piazza is a wonderful tool, very well-designed and attractively laid out. But I suggest using it sparingly. I'll give an example in class about this.

January 29, 2:55 p.m.

One of the topics we'll cover tomorrow is disassembly of Python byte code. Our coverage will be brief, but the general point to be made here is that scripting languages tend to be slow, so looking at the generated assembly language can help us modify our Python code for faster execution. (Note, though, that this is still code for the Python Virtual Machine, not native code for the machine we are running on. Interpretation of the PVM code will still involve some overhead.)

As I have mentioned before, the Julia language, an MIT project that has attracted quite a following, is a new scripting language that is intended to combine the high-level expressiveness of scripting languages with fast execution. As such, it is interesting, as I noted, that Julia's interactive mode allows one to directly inspect the native assembly language.

For instance, blogger Leah Hanson has this example:

function linear_foo()
  x = 4
  y = 5
end

julia> code_native(linear_foo,()) # returns the value of y
        .text
Filename: none
Source line: 3
        push    RBP
        mov     RBP, RSP
        mov     EAX, 5
Source line: 3
        pop     RBP
        ret

which you may recognize as Intel assembly code.

Compare this to Python and R. The former was aimed at elegance and OOP. R's main language philosophy is functional programming, which purists love but inhibits performance. (Python includes some optional functional language features, but is not a functional language.)

January 29, 1:55 p.m.

As mentioned, the coverage of any quiz technically consists of all the material covered in class so far, but in practice usually extends only through the most recent Friday lecture. For tomorrow's quiz, for instance, that means Chapter 1 through p.39, and all of Chapter 2.

January 27, 11:45 p.m.

If you received a -1 grade (9 students did), please reformat according to the rules in the syllabus and e-mail to the TA for grading.

January 27, 10:05 p.m.

Your quiz grades are being e-mailed to you as I type this. The solutions are available in the Exams/ directory of our Web page.

A note on Problem 2: The answer was sprintf(), not printf(). Both in class and in the book (p.17), it was emphasized that, as it says in the book, this "is a string issue, not a print issue." It is a technique for string manipulation.

Please note that I graded Problem 4 quite liberally this time. If a student had the right function but wrong argument, I gave full credit.

January 27, 9:25 p.m.

Just to make sure we are all on the same page: The assignment specs simply state what output one gets for what inputs. You just need to make sure the TA's test script works correctly on your code. Other than that, there is no restriction on what you do.

January 27, 11:15 a.m.

A couple of people have asked me about placing your data structures in the same file that stores the fs files, i.e. the file referred to as f and fsname in the assignment specs.

Actually, the specs do not allow this. They say,

Upon suspension, all data structures etc. are saved to a native file whose name is the concatenation of the name of f and '.fssave'. The saving/resuming of the system is done via the Python pickle module.

So, say you create your file system by calling

fs.init('abc')

and then create fs files x and y. Those two files will be stored in abc. But you will also have data structures that keep track of free space, chunking of files and so on. While fs is running, you'll have these data structures in memory, but when you suspend your system by calling fs.suspend(), those data structures will be "pickled" and saved to a native file 'abc.fssave'.

January 25, 8:15 pm

Here is some advice I gave to one of the homework teams in my office hour today:

First and foremost, I told them to START SMALL. I mean, REALLY SMALL. This is my philophy of programming, and trust me, it works.

Now, below is a scenario that will help you understand what the system is supposed to do, and one you can use for testing after you write a minimal fs:

Again, take Baby Steps. Everyone who has asked me about the homework has asked about directories, in spite of the fact that I recommend that you get the thing fully working WITHOUT directories first.

LATER, when you do add the directories feature, keep in mind the following. A directory (in general, e.g. Linux and Macs, not just in fs) is just an abstraction. There is actually no physical "tree" structure to your files on disk; instead, directories simply relate groups of files. So, in your fs, you might just implement a directory as a Python list of strings, say ['q','r1','aabb'] for representing a directory containing files q, r1 and aabb. Of course, you'd also need a similar list for subdirectories of this directory. This is just an example; there are lots of approaches you could take. (Unix-family OSes treat directories as special files.)

January 25, 8:15 pm

I'd like to encourage everyone to attend, and better, participate in, the iidata conference on Data Science. This event is on campus, and is entirely student-run. Last year, the first event, was a huge success. I believe that many attendees didn't realize that it was a student-run event, it was so professionally done.

January 25, 3:40 pm

In the text, it mentions several times that the split() method in the string class has a blank as the default separator, but that one can specify a separator as an argument. Examples:

>>> x = 'abc.de.f'
>>> x.split()
['abc.de.f']
>>> x.split('.')
['abc', 'de', 'f']
>>> y = 'abc de f'
>>> y.split()
['abc', 'de', 'f']
>>> y.split('.')
['abc de f']

Note by the way that all contiguous instances of the separator will be treated as one, e.g.

>>> z = 'abc de  f'
>>> z.split()
['abc', 'de', 'f']

January 25, 2:55 pm

If your homework group size falls below 3, inform the TA immediately.

January 24, 11:30 pm

I'm still getting questions as to what the basic operation of fs is supposed to be.

Suppose you have a file abc on CSIF. To make things even more concrete, say the file is /tmp/abc on pc28. (I choose /tmp because most of the file system on CSIF machines are shared, physically present on a different machine.) You will be storing private files in the space occupied by abc, the native file. The Linux OS running on pc28 won't know about those files. It just knows about abc.

The fs system administrator (including you, when you test your code) creates abc before creating the fs file system. Of course, the bigger abc is, the better, as it will give users more space. The initial contents are irrelevant.

I strongly recommend that, while fs is running, you keep a copy of abc in memory. Upon shutdown, you write the current contents back to disk, by writing to abc. Upon bringing fs up again, you read from disk to memory. Those of you who are really sophisticated in systems matters might even try mmap().

Part of the value of this assignment is that it will get you to think about things that you've taken for granted before, such as: What is a file system? What is a directory? Don't hesitate to ask me or the TA about such things.

January 24, 12:25 pm

Please note that your fs module will be just that, a module, basically a library. It will consist of functions, and likely some classes (depends on your approach), but it will not be an executable program. The user (including you, when you test it) will write the executable code, which makes use of your library.

January 24, 8:20 am

"Haste makes waste." Already had fs.deldir().

January 24, 8:20 am

A student pointed out that I had neglected to include a function fs.mkdir() in the specs. I've now done so, along with fs.rmdir().

January 24, 12:20 am

News on the homework:

January 23, 8:20 pm

In quizzes, please copy your answers to a separate sheet AS YOU GO, not at the end of the quiz period. Also, please do not take pictures of your quiz; sorry, but no electronic devices are allowed at any time. (If later on you need to see your quiz, due to your asking me for more credit, you can get a picture of your quiz from the TA.)

January 23, 2:35 pm

In response to some questions I've received on the programming assignment:

January 22, 8:35 pm

Tomorrow's quiz will cover the material through p.16 (i.e. through Sec. 1.5.3.2) and all of Chapter 2.

Tomorrow's lecture will resume Chapter 1, starting with Sec. 1.5.4.

January 21, 5:35 pm

Here are a few examples of string manipulation with non-ASCII/non-visible characters:

>>> x = ''
>>> x = x + 'a'
>>> x
'a'
>>> x = x + chr(33)
>>> x
'a!'
>>> x = x + chr(10)
>>> x
'a!\n'
>>> print x
a!

>>> x = x + chr(255)
>>> x
'a!\n\xff'
>>> len(x)
4

January 21, 5:30 pm

I deleted a couple of the more intense parts of the assignment. :-)

January 21, 2:50 pm

The day you have been anxiously awaiting is now here! Your first programming assignment is now ready!

I believe you will enjoy this assignment, and learn a ton from it. However, there is a lot of work to do, so teamwork will be of the essence. DON'T UNDERESTIMATE THE WORKLOAD ON THIS PROBLEM! Fortunately, this will be the type of project in which it is fairly easy to subdivide the work among team members, but I'm predicting that some teams will put the work off too long and will not finish.

I probably will need to issue a small tweak or two to the specs here, but the assignment is ready to go.

January 21, 10:30 am

Though it is implied, I wish to make it explicit: Any information disseminated on the blog is considered part of official course materials. This means for example that it may be needed in quizzes.

January 21, 9:00 am

I will be writing up your first programming assignment, Homework I, sometime today. It probably will be ready by early afternoon.

January 20, 12:25 pm

Our procedure for taking quizzes is the one described in the ECS 132 syllabus. In order to ensure that you get credit for your quiz, make absolutely sure you understand this procedure in full detail. You may wish to do a test upload to handin.

January 18, 11:55 pm

Please keep in mind what I stated in my prequarter message (and repeated on the first day of class):

You could decide to skip lecture and just read the book, but it IS really helpful to go to lecture. There will be spur-of-the-moment examples and insights, answers to students' questions. etc. And you are responsible, in Quizzes and interactive Homework grading, for everything that comes up in lecture.

January 13, 9:10 pm

I refreshed my memory on the extend() method. Here is the issue: The append() method adds one element to the existing list, while extend() concatenates a new list with the existing one. Consider the following:

>>> y = [1,2,3]
>>> y.extend([8,88])
>>> y
[1, 2, 3, 8, 88]

Here we concatenated [1,2,3] and[8,88].

But look at a more subtle example, alluded to by John:

>>> x = [1,2,3]
>>> x.extend('abc')
>>> x
[1, 2, 3, 'a', 'b', 'c']

As explained on p.15 of our book, a character string is considered a tuple, i.e. a read-only list. So, the string 'abc' is a 3-element list, basically ['a','b','c'] but read-only. Now you can see why the above pattern emerged.

By contrast:

>>> x.extend(['abc'])
>>> x
[1, 2, 3, 'a', 'b', 'c', 'abc']

January 11, 8:00 pm

My office hours will be MW 5:15-6:30 pm and by appointment.

Monday, January 11, 7:55 pm

On Monday I said that I wasn't sure whether we will have a quiz next week. I didn't realize that next Monday is a holiday. Since our discusssion section is on Mondays, this means no quiz next week (and on Feb. 20, Presidents Day).

January 10, 7:15 pm

Please bring your hard copies of the textbook to class, starting tomorrow. I ask that people not use laptops in class, other than at the back of the room, and in any case, you will need a hard copy of the book for the quizzes, which may begin next week.

In general, I suggest that you do the reading associated with a given lecture after the lecture rather than before.

Keep in mind that:

Monday, January 9, 9:35 pm

Welcome to our class blog! Please remember that it is required that you read the blog at least per day.