Archive for WebCourse

Robotics Makeover

This year I am revamping both web development and robotics courses I teach during the summer.

  • Web Development
    • Up until now I have been using Plone/Zope/Python for both the course website and teaching web development. Lately though I have been examining Visual Web Developer, ASP.Net, and C#. While I think Plone/Zope has its advantages, clearly ASP.Net is more popular, and its feature set (especially with 3.5) is getting to be pretty comprehensive. What has finally convinced me to migrate is the fact that ASP.Net pages can now be written in IronPython, the .Net implementation of CPython.
    • IIS seems to have its issues, however, so I am looking into reverse proxies that will hide IIS and Plone/Zope behind it.
  • Robotics
    • I have been using Parallax BASIC Stamps, Sumobots, and Cypress Semiconductor PSOCs. While the Sumobot is a very nice hobby robot, the computational power of the BASIC Stamp that is at its core leaves a lot to be desired.
    • Microsoft has the Robotics Studio, a .Net platform for robotics work. It has gained sufficient support that I think it is a viable platform on which to teach my course. It requires an interface on the robot that is too much for the Sumobot (although there is sample code for the BOEBot, which is similar). I have been looking at the ARM series of microcontrollers for some time now, and have decided to have a hybrid solution, where a PSOC handles all the low-level sensor/motor interfaces, and the new ARM Cortex M3 handles the AI/machine learning as well as the interface to the Robotics Studio.
    • I purchased the Olimex STM32-P103 prototyping board, based on the ST STM32F103RBT6 Cortex 32-bit microcontroller. It has the following features, from the Olimex website.
    • ARM 32 bit CORTEX M3â„¢ with 128K Bytes Program Flash, 20K Bytes RAM, USB, CAN, x2 I2C, x2 ADC 12 bit, x3 UART, x2 SPI, x3 TIMERS, up to 72Mhz operation.
    • I think I have gotten spoiled by the rich set of peripherals available on the PSOC, because while the set of peripherals the Cortex has is reasonable, by itself it will not be able to manage all the sensors and hardware of a robot.
    • I am thinking of using a new chassis for the robots, I need to do a full engineering preliminary design before I can commit to that course of action.

After much reading and searching I have finally managed to burn, run and debug a sample LED blinking program on the STM32-P103. The beauty of open source development tools is obviously the cost and the frequent updates, but the disadvantage is the lack of good documentation or manual. I have been able to piece together bits and pieces through a lot of trial and error, and will be posting everything I have learned so far soon, with sample code.

Comments off

UN Data Mining

The UN has made data it collects from its member nations available here. Very interesting, and I am looking forward to mining some of the data to prepare for the web course this summer, as well as try my hand at some economic analyses.

Comments off

Internet Explorer CSS Box Bug

It is no secret that IE has some bugs in rendering CSS styles, most notably with respect to the box model. This wikipedia entry has a good summary of one of the bugs, which is really a misinterpretation of the CSS standard. It doesn’t help that for some web developers, Microsoft’s interpretation makes more sense than the “correct” standard!

Comments off

Database Connection Security!

I just caught a very important security issue that I had missed! The SQLPAS PlonePAS module requires a database connection to the user/password tables. Since you can’t add a Z MySQL DB Connection (or any dbc for that matter) in the acl_users folder, I left it at the top level root directory. This unfortunately has the effect of making that database connection usable to students in the Members directory, by acquisition! They could conceivably then do a SHOW TABLES, etc to find out student info, including grades!

The solution was to create a Z MySQL DB Connection with the exact same id and create it in the Members folder. That way, if a student creates a Z SQL Method the only connection that he/she sees would be the one to the test database, not the student info db!

Comments off

Workflow Finally Solved!

Well I finally have it working the way I want.

Rundown of flow:

  • Student creates PSFolder
  • Student submits PSFolder
  • PSFolder gets moved to separate folder that only TFs can access
  • TF logs onto account and goes to tf_resources and selects View Submitted Homework
  • TF grades each submitted PSFolder
  • When done, TF returns to student PSFolder
  • PSFolder gets moved back to student folder

New features to be added eventually

  • Incorporate grading pages to enter grades while in workflow (this is complicated)
  • Incorporate archiving of submitted folder in case of disputes later on
  • Email on homework submission/return

Some hard lessons learned

  • If you manage_pasteObjects(manage_cutObjects()) requires both Copy or Move and Delete permissions (need to confirm)
  • manage_cutObjects() takes a list (or tuple?) of ids, not an id. (sort of obvious)
  • portal_catalog gets confused if you do a copy -> paste. do a cut -> paste. Confused means the review_state variable is “private” (the original state) not “submitted”
  • if as a student you can no longer see your home folder, that means some workflow permissions were screwed up, just reset security settings in portal_workflow to clear this up
  • put the object move script in the After section, not Before. otherwise portal_catalog gets confused too, as the state hasnt changed to submitted yet.
  • Properties, based on a db using SQLPASPlugin, need to be set through Plone using setMemberProperties, not directly into the db.
    • I created an external method script to do this. extSetMemberProperty // extSetStudentTFStatus

Parts of the workflow system

  • Portal_catalog: Add index/metadata for TF
  • Portal_workflow: Add PSFolder type
    • Add Variable TF
    • For submit transition use expression: TF = python:here.portal_membership.getAuthenticatedMember().getProperty(‘tf’)
    • Set Script (after) to an external method that handles the move extMoveObjectsPlone
    • For return transition set script (after) to extMoveBackSubmittedPlone
    • Both are in the external file extCopyObjects.py
    • Permissions for the submitted state: Owner should have Access contents, Copy or Move, and Delete
    • You need to edit the Adds to actions box and add a label or else the transition won’t appear (e.g. “Submit to TF”)
  • PSFolders are Plone objects, not ZMI/zclass objects. Create a regular Plone folder to store the moved psfolder objects in the top level portal root, and manually create Plone folders with tf user ids as ids. Make sure this folder is Private!
  • Set permission to private, submitted, graded, etc appropriately.

I’ve added code snippets to the svn server, which can be accessed on trac.reisun.com

Comments off

Workflow Madness in Plone

I’ve been having some difficulty implementing student homework submission. I’ve gotten a crude hack working, but trying to improve it has been more difficult than I imagined. The following excerpt from http://www.zopelabs.com/cookbook/1029298314 will help, I wanted to copy it here so I can refer to it later. (For a while I had forgotten how to access this page!)


I used your idea but ran into a problem because the default action after a change in status is to view the object (at the old location), which results in an object-not-found error.
To fix this, I went into portal_properties/navigation and changed default.content_status_modify.success from "action:view" to "url:../folder_contents" Now when I publish an item, it dumps me into a view of the folder where it came from.It would be nice if it would go to a view of the newly-moved object, but I haven't been able to get necessary redirects working.Fixing Plone's response (was Re: Re: Use one script for many types of objects.) by sh23 - 2004-10-14
I wanted to only be placed in the directory when the object was no
longer there. In /Plone/portal_properties/navigation_properties
instead of modifying default.content_status_modify.success, I created
a new entry:

default.content_status_modify.gone url:../folder_contents

I then modified (a custom copy of)
portal_skins/plone_scripts/form_scripts/content_status_modify by
replacing 'success' with status_string in the return statement, and
then adding the following immediately before the return statement:

if context.restrictedTraverse(context.getPhysicalPath(), default=None) == None:
status_string = 'gone'
else:
status_string = 'success'

This is known to work with CMF 1.4 and Plone 1.1 24 June development version.

Comments off

Plone PAS NOT hooked up to portal_membership?

I have been trying to convert my course website code to work with the new Plone 2.5.2 install. Typically, new features aren’t documented clearly or at all. The biggest hangup is the PlonePAS setup, that has changed the way membership data is accessed. No longer do portal_membership calls work, or at least not completely.  For example, context.portal_membership.listMemberIds() works, but only for the users defined in the default member plugin. I use SQLPASPlugin, and the above code does not list members defined in an SQL database.

The only way I’ve been able to get access to them programmatically is to use context.acl_users.X which I am not sure is the right way. Some test code below:

# Test code for new PAS Plugin architecture---NOT hooked into portal_membership
# print "Current User: ",context.portal_membership.getAuthenticatedMember()
# print "Users: ",context.portal_membership.listMemberIds()
# print "Search Users: ",context.portal_membership.searchForMembers(name='staff1')[0].getRoles()
# print "Search Roles: ",context.portal_membership.searchForMembers(roles='cscishstaff')
# print "acl_users: "
# users=context.acl_users.searchUsers(roles='staff')
# for u in users:
#     print "%s(%s)"%(u['userid'],u['title'])
#     print u
# return printed

Comments off

Saga of ZMySQLDA on Plone 2.5.x continues

I had hoped that I was done with installing ZMySQLDA on Plone, but it turns out there were some additional hiccups. ZMySQLDA was being recognized as a product in the Control Panel, but it disappeared from the drop down menu again. Looking at the event.log revealed that it was trying to extract files from an egg file into my personal user home directory, not the “plone” user that was created and under which the plone process was being run. Not exactly sure why this is happening, but I took the lazy way out and manually copied and pasted the MySQLdb folder and _mysql.so file into the Python directory. An added complication was that the _mysql.so had to be also copied into the ZMySQLDA directory.

 I really need to sit down and figure out all this egg and process configuration out, but am running out of time setting up for my summer courses.

Comments off

XSS Cross Site Scripting Resources

I have been ignoring going into details of web security in my introductory web class, partly because we cover so much (HTML, CSS, Javascript, SQL, python) that we really don’t have time to cover it, and partly because I’ve been too busy to come up with some slides for it.

With the prevalence of XSS (Cross Site Scripting) hacks, it looks like this year I will be spending at least a few slides on security.

Some resources:

An informative post on slashdot outlined a few points to consider when designing a site to be robust against XSS (I’ve highlighted his points, but the words are mostly his):

by Fireflymantis (670938)on Friday June 15, @04:21AM (#19516455)

  •  Are you making sure, without fail that if a user changes view.php?id=32 to view.php?id=33 that they are not getting access to content they shouldn’t be?

  • What about cookies? Assuming the malicious user can (and will) build cookies of their choosing and content, are you making sure that this cannot somehow be used to hijack another users account?
  • Are you 100% certain, that every time you read get/post/put data that it has been marked as tainted, validated, and only after it has made it through some very harsh sanity checks it is allowed any where /close/ to a DB insert/query?
  • It gets even more muddeled in the world of XmlHttprequests when you have to validate against a plethora of other constraints
  • Simply checking form data is almost 99% of the time not enough. For a non-trivial web app, even the above is not easy to do unless you pay attention to it every step of development. And even if you do that, you will probabaly miss something.

Comments off

ZMySQLDA on Plone 2.5

I’ve just spent the last 2 hours or so installing ZMySQLDA on a Plone 2.5.3 site. It was considerably more painful than I thought, as I never used to have troubles with this product before.

The problem? PIL is required to be installed for Plone, and its ImageFile package conflicts with a similarly named package that is part of Zope (App.ImageFile)

 This is fixed in the DA.py file of ZMySQLDA by explicitly importing the App.ImageFile package, not the PIL version.

 After having the ZMySQLDA product finally appear in the Control Panel, it still wasn’t showing up on the drop down “add” menu. I am not sure what I did to fix this, but for a while only a single ZEO client had the menu option visible. Although I had rebooted/restarted the ZEO client/server multuple times and multiple ways (through the control panel and using the restartcluster.sh script) I think I still had the problem.

In anycase, it now works and I have both ZEO clients showing “Add Z MySQL Database Connection.”

I had planned on getting SQLPASPlugin setup, but it is already 1227, and I need to go to bed.

Comments off