IRC log of #schooltool for Monday, 2008-10-20

*** povbot has joined #schooltool03:16
*** povbot` has quit IRC03:32
*** Aiste has joined #schooltool07:03
*** povbot has joined #schooltool10:05
*** ignas has joined #schooltool14:40
*** jelkner has joined #schooltool15:37
*** nubae has joined #schooltool16:06
nubaehey, the link to translating leads to nowhere:
nubaeFolks in Afghanistan (big deployment of 20,000 xos) are considering using schooltool, and asked me about localisation in Pashto, I suppose they'll do this through launchpad, am I right to assume right now only localisation in German has been done?16:13
nubaestrike that, just found the other languages16:15
*** th1a has joined #schooltool16:23
*** dwelsh has joined #schooltool16:23
th1aGood morning/afternoon ignas, aelkner, yvl, dwelsh, jelkner.16:30
th1aGo Phillies!16:31
jelknerhappy monday16:31
ignasis not in here16:31
th1aSo we're 10 days out from release.16:31
th1ablockers or potential blockers ignas?16:32
ignasubuntu packages backing up database16:33
ignasand gradebook evolution16:33
ignas1. - i haven't done it in my life, so i can't estimate it16:33
th1aYes... I guess we need to get jinty in on this if possible.16:33
ignas2. - well, i have looked at the abyss and it's looking back at me16:33
th1aWhich abyss?16:34
ignasschooltool.gradebook and schooltool.requirements16:34
ignasi am trying to very very quickly understand what where and how is going on in there16:34
aelknerrequirements are now no more than ordered containres16:35
ignaslike - what are the implications of gradebook plugin being enabled/disabled and then enabled again16:35
th1aWe should probably spend some time discussing that, but first finish getting quick reports.16:35
aelknerwe hve the AppInit issue16:36
th1aLet's get aelkner's report first.16:36
ignasaelkner: what kind of?16:36
aelknersorry, i deidn't mean to interrupt16:36
aelkneri saw marius' post to my mail-queue issue16:37
aelknerit could be that using a relative path in the config file is the problem16:38
aelknerbut i need chris to be available one of these evenings16:38
aelknerto coordinate bringing down the server, changing the setting16:38
aelknerand having him send an email to see if it works16:38
th1aCan't you send yourself an email?16:39
aelkneri thought of that, but i would have to add myself as a teacher to their system16:39
aelknerand i don't want to touch their data that way16:39
th1aI'd just do that.16:39
ignasor set the email for "schooltool manager"16:39
aelknerok, i can do that tonight16:40
aelkneri saw the chat on Friday about the person container filter widget16:40
aelknerso i changed cando to override that widget16:40
th1aThe thing with Chris is that it is probably best to send him an email saying "I'm going to do x, y, and z on the server tonight" and go ahead and do it unless he suddenly objects.16:40
aelknerand i got rid of the use group.members16:41
ignasaelkner: did that affect anything?16:41
aelknerit might have sped up things some, but it still takes sconds to load16:41
ignasi still can see the counts of students16:42
aelknerthere are 9000+ persons16:42
ignasin acc08.candoskills16:42
aelknerand the mere act of getting that count16:42
*** jstraw has joined #schooltool16:42
th1aThis particular issue doesn't affect actual users though, right?16:43
ignasit's still "Students (9532)" in person filter widget16:43
jstrawhi all16:43
*** jstraw is now known as Lumiere16:43
th1aI mean users don't have a reason to go to that index?16:43
ignasth1a: it does, apparently cando users are using that view to access specific students16:43
aelkneryou're talking the live cando instance16:43
ignasaelkner: yes16:43
aelkneri need jstraw to update that16:43
ignasoh, you tried it on a test instance16:43
aelkneri also added the report menu items that dwlesh was looking for16:44
th1aHow long does it take to search for an individual student?16:44
th1a(i.e., after you hit the search button)16:44
dwelsh5-7 seconds16:44
th1aSame as when you're just loading the index?16:45
dwelshok.  that's a good question.  checking16:45
th1aBecause, there is no real reason to load that list of students at all, other than reminding new users what the page is about.16:45
dwelshonly 1-3 secs. to search a person16:46
th1aIt is more of a navigational hint than a useful thing.16:46
aelknerth1a: what do you mean by "loading the index"?16:46
th1aSo just axe out the list and only have the search box there.16:46
dwelshmore like 1-2 secs.16:46
th1aLoading the index page.16:46
th1aWhat I'm saying is that having 10 student names or whatever on the persons index is mostly useless.16:47
aelkneryeah, that's the page i fixed16:47
dwelshthat's a good fix; the list of ten students is not usefull16:47
th1aThe only reason it is there is to make it a little more clear what the page does the first time you're using SchoolTool.16:47
aelknerth1a: you're saying the ten students that are listed on the first page of he index would slow down the loading?16:48
th1aOr if you have a really small school.16:48
th1aThat's the entirety of the problem as far as I know.16:48
dwelshthat makes sense16:49
th1aOr maybe I'm confused.16:49
dwelshno, I think you've got it.16:50
th1aAre we talking about the student group index or the persons index?16:50
ignaswe should profile that view16:50
ignaswell not we, alan should ;)16:50
dwelshin CanDo, we're talking about the "Enrollment" tab16:50
aelkneryeah, how does one profile the view?16:51
dwelshThe view that tab brings up lists current students and "available" students16:51
aelkneri didn't change that one16:51
dwelsh"available students" is the beginning of the 9K+ person list16:51
dwelshthat beginning of the list is not useful16:51
dwelshand teachers have to use that page a LOT.16:51
dwelsh(every time a student adds or drops)16:51
dwelshso it's very tedious for them16:51
th1aCut out the list then.16:52
aelkneri could look into optimizing that one too16:52
th1aAny other pressing concerns dwelsh?16:52
dwelshaelkner, did you say the two reports (VA and Teacher) are now fixed?16:53
aelkneryou need to reload cando, svn up it16:53
dwelshlumiere:  maybe you could do that today?16:53
dwelshor show me how to do it.16:53
aelknerLumiere: i wanted to do it myself last night16:54
aelknerbut i wasn't sure how you're starting the server these days16:54
dwelshI also put an email out to all CanDo08 users asking for url's of any other pages that load unacceptably slowly.16:54
dwelshI'll be watching the speed issues as well.16:54
dwelshAll's quiet as of this am, but we maybe be getting more input as the week progresses.16:54
th1aOK.  Cool.16:54
Lumiereok, I will update cando today16:55
th1aLets go back to discussing gradebook/requirements.16:55
dwelshthanks.  so then we're looking good.  I'll check reports as soon as we svn up.16:55
dwelshThanks, SchoolTool team, for support at end of last week.16:55
th1aFirst, essential point:  We don't generally have to worry about migrating gradebook because it hasn't been released before.16:55
dwelshWe were in a bit of a crisis there with users, you you guys helped a lot16:55
th1aYou're welcome, dwelsh.16:56
th1aGood work ignas and yvl on picking up the crazy DNS bug.16:56
ignasth1a: alga helped too16:56
ignasth1a: yvl and I looked and asked "WTF?!" and alga gave us the answer16:56
th1aIt is good to have the collective brainpower of POV in reserve.16:57
aelkneris that obama having powell?16:57
aelkneris that like16:57
th1aSo... actual gradebook users: jelkner, I guess SLA.16:57
ignasth1a: well - it has been - cando was released, so if you want them to be able to use schooltool with school years, i will have to look at it16:57
th1aaelkner: Not really.16:57
aelknerthey're still both really good to have16:58
ignasbut both cando and SLA are not using terms16:58
th1aWell, that's a different question itself, right?16:58
th1aCanDo is a different question.16:58
ignasyeah, i mean - i can "not break" cando quite easily16:58
ignasthe problem is - not breaking "th1a school setup"16:58
ignaswell - you have mentioned that you are using gradebook and terms16:59
ignaswhich is the tricky case in the whole story,  because migrating can do does not require me knowing anything about gradebook16:59
ignass/can do/cando16:59
th1aI am using gradebook and terms?16:59
ignaswell - aren't you?17:00
th1aI think we may have had a misunderstanding...17:00
ignas"Yes, Ignas, I am trying to use the Gradebook with timetables.  What's17:00
ignasthe problem with that?"17:00
th1aWhen did I say that?17:00
ignasOct 17 (3 days ago)17:00
th1aWell, that is, I was trying to check out some bugs in the gradebook.17:01
ignasahh, I see17:01
th1aThat was the context I was answering in.17:01
* ignas thought "Must save th1a's database"17:02
th1aNo, not at all.17:02
th1aFor SLA perhaps aelkner should just write some import/export scripts for grades.17:02
aelknerhow so?17:03
aelkneri'm not sure i understand the issue17:03
ignasi can try and explain17:03
ignasSections could be scheduled for multiple terms.17:03
ignasNow - they can't and there is 1:1 relationship between a section and a term.17:04
ignasSo if you had 3 terms and 1 section, evolution script will make it 3 terms and 3 sections.17:04
ignasThis means that - if section had 3 timetables, timetables will be assigned to the relevant sections (easy to do, timetables have attribute "term")17:05
ignasCalendar events will be moved to relevant sections (easy to do - there are dates of the events that fit into term date ranges)17:05
ignasGrades - are tricky, data structure is a mess, and i can't move them around in any straighforward way.17:05
ignasSo if there were grades for a section - i have no way to know which "term" these grades are for.17:06
aelknergrades are not stored in the section17:06
ignasthat is a part of the problem17:06
ignasthat makes it so difficult17:06
ignasAnd i have no easy way of duplicating the gradebook, to allow people to delete the sections/worksheets they don't need.17:07
ignasthough - if there are no terms17:07
ignasor there is only 1 term17:07
ignasi can "move" the section17:08
ignaswhich will preserve all the grades...17:08
ignasso cando - will work, they don't have terms17:08
ignasschooltool.stapp2008spring will work, they don't have gradebook17:08
ignassetups that have more than 1 term and are using schooltool.gradebook17:08
ignasare very very tricky17:09
th1aignas: Do you the sections of the same class over multiple terms know they are related to each other?17:09
ignasth1a: at the moment - no, i have it in my TODO list, but i am not sure I can make it into the release...17:09
ignasth1a: it will be my first thing to do after the release i think17:09
th1aThat's the real problem.17:09
th1aIt seems to me that there really needs to be an object that describes the multi-term section and the gradebook connects to that.17:10
ignasthere will be, and schooltool.lyceum.journal does that anyway17:10
ignasbut i will have to add that functionality to schooltool and17:11
ignasmake schooltool.gradebook use it17:11
th1aSo let's worry about forward compatibility rather than backward.17:11
ignasis SLA using terms and timetables?17:12
th1aTerms, not timetables, right?17:12
ignaswell - terms17:12
ignasaelkner: ?17:12
aelknernot at the moment17:13
ignasthen - we have no problems with know deployed instances17:13
ignasCando - will work, SLA - will work, stapp2008spring - will work17:13
th1aThat SchoolTool works at all without terms is just wrong.17:13
ignasth1a: well - it will stop working soon17:14
th1aYes ;-)17:14
ignasWithout terms that is ;)17:14
ignasand then - i will need a break to get some rest and play with some pet features like "full school year export" (because I would like to do that ;)17:15
LumiereI think it might be good pre-final release17:15
Lumiereto have some sort of startup druid17:15
Lumiereto walk you through setting up a term/timetable etc17:15
ignasLumiere: we know, we have spent half of the EuroPython sprint talking about it17:15
Lumiere´╗┐(hookable so cando can ask them to setup a scoresystem and import initial competencies17:16
th1aSo in the migration between beta and 1.0 we'll be adding multi-term section objects, but we'll probably just attach each section to one multi-term section object, for the sake of sanity.17:16
ignasnope, i will just add attributes17:16
ignas"next" and "previous" to sections17:17
ignasso gradebook does not have to look them up17:17
ignaseasy to do, and simple enough for programmers to understand17:17
ignasso you would just get a section for a person when displaying gradebook, and show links to the previous term grades, or show all the related sections in the same view17:18
ignasand I can do that quite quickly17:18
ignaswith support for "copy all the sections to the next term" in add term view17:19
ignasquickly as in - 2 days17:19
th1aOK.  But do you intend to add  multi-term section objects after the release?  I'm confused.17:20
ignasno, i don't intend to add a different object just for that usecase17:20
ignasi don't think we need it to fulfill the usecases17:20
th1aI will resist the temptation to think up obscure potential future use cases.17:21
ignasth1a: you can, i will keep them in mind17:21
ignasand it might help me improve my plan17:21
th1aEspecially 10 days before a release.17:22
ignasyeah, it's better to do that 10 days after the release ;)17:22
th1aIt just might make report generation harder to write.17:22
ignasth1a: not much17:23
th1aBut a few helper functions is probably easier than adding a basic object and rearranging everything.17:23
ignasyeah, and then making people *think* about both kinds of sections17:24
ignasall the time they write code17:24
th1aSo at this point should we put aelkner on the index speedups?17:25
ignasaelkner: do you know how to profile things?17:25
aelknernot yet17:25
aelknercould you explain briefly?17:25
LumiereI would like to know too17:26
ignaswell - it's an art not a science, but I can try17:26
ignasgimme a second17:26
* th1a begins profiling ignas's performance.17:26
th1a47 seconds to get more coffee,17:27
th1a64 seconds to pee,17:27
* th1a goes to get more coffee.17:27
Lumiere´╗┐ignas: the more of us know about it, the more who can test this17:27
ignasstep 117:27
ignasLumiere: it's not just about finding out what is slowing it down17:28
ignasLumiere: it's also about knowing what to speed up, and how17:28
ignasstep 2 is - finding the __call__ of the view17:28
ignasand if it does not exist - adding an __call__ method that calls self.index()17:28
ignasand returns it17:28
ignasand then @profile'ing it17:29
Lumiereignas: yes, but if I can at least find out what is slowing it down, I can at least let someone know17:29
ignasoh and then you get a report and try spotting the hot spots, abnormal stuff, broken algorithms, useless operations and stuff17:29
ignasLumiere: are you talking about profiling17:29
ignasLumiere: or about timing?17:29
Lumiereboth really17:30
LumiereI am interested in both17:30
ignaswell - i don't know how to hook up timing to zope3 app17:30
ignasLumiere: i'd do it using WSGI17:30
ignasand a log file17:30
Lumierewhen I get a chance I'll probably look into it17:31
aelknerignas: if i make a change to use @profile as the page suggests17:33
th1aaelkner: Does that help?17:33
aelknerdo i still start the server with make run17:33
ignasstart the app17:33
aelkneri.e., bin/start...17:33
ignasrefresh the view17:33
ignasthen Ctrl-C the app17:33
ignassee results17:33
ignasyou might want to add a delay17:34
aelkneronly after ctrl-c does the info spill out?17:34
ignasto allow the cache to warm up17:34
ignasonly when the process is over17:34
aelknergot it17:34
aelknerdealy for cache?17:34
aelknerhow do you mean?17:34
aelkneri meant delay17:34
ignas@profile takes a parameter count17:35
ignasi think17:35
aelkneroh, a delay there17:35
ignaswhat it does is - it skips the first few runs17:35
aelknergot it17:35
ignasof the function17:35
ignasso you refresh it 2-3 times17:35
ignasand only then it starts counting17:35
ignas(first run can be slower because of database loading and stuff)17:35
ignasi'd start from profiling of SectionLearnerView17:35
ignasbecause it is the view that CanDo users are using the most17:36
th1aOne thing to emphasize the next couple weeks is being available on IRC.17:36
th1aI'll try to stay close and keep my eye on it..17:36
th1aYou guys can keep talking, but I'm going to...17:37
* th1a drops the bag of gravel.17:37
ignasaelkner: just add:17:37
ignas    @profile17:37
ignas    def __call__(self):17:37
ignas        return self.index()17:37
ignasclass SectionLearnerView(RelationshipEditConfView):17:37
ignasand refresh the view a couple of times17:38
ignasit will be way slower than usual17:38
ignasbecause of profiling17:38
ignasthen ctrl-C the app17:38
ignasand lisppaste5 the output ;)17:38
Lumiereaelkner: if you want to work with the actual server, let me know17:38
aelknerLumiere: no need17:38
ignasor try deciphering it yourself17:39
aelknerignas: i'd have to do that to the schooltool egg17:39
ignasyeah, egg is good17:39
aelknerbecuase cando does not use a checkout17:39
ignasas long as you know how to revert your change17:39
Lumiereaelkner: you should be able to sftp a data.fs whenever you need17:39
aelknerignas: yeah, i needed to remove the egg and bin/buildout to get it back17:40
ignasjust delete the lines you added17:40
ignasworks for me most of the time (as long as I can recall what I did)17:40
aelkneryeah, that would be the best way17:40
aelknerbut i lost track of the change i made last time17:41
aelknerand lost the undo becuase i restarted the editor17:41
* jelkner stays logged in to channel, but leaves to await arriving students...17:42
LumiereI suggest copying the egg17:42
Lumiereand running diff ;)17:43
Lumierediff -ur dir1 dir217:43
aelknerignas: it didn't like my include of profilehooks17:46
ignaswell - put profilehooks in your pythonpath17:46
ignaslike - in the same directory like the view17:46
ignasand then give it a path like schooltool.course.profilehooks17:46
aelknerignas: i don't know if the change to __call__ is right17:52
aelkneri'm getting a traversal error17:52
aelknerTraversalError: (< from /home/aelkner/.buildout/eggs/schooltool-2008.10.0dev_r2436-py2.4.egg/schooltool/course/browser/templates/ object at 0xa6e3b8c>, 'batch')17:52
aelknerRelationshipViewBase has __call__ = ViewPage...17:54
aelknerso overriding it would not work, right?17:55
ignasvery strange17:55
ignasin that case17:56
ignasinstead of self.index()17:56
ignasdo the RelationshipViewBase.__call__(self)17:56
th1aignas: btw, you've got about 10 days to pick up your Sansha's Radio S and Sansha's Radio M in Rens.17:57
ignashmm, i haven't seen the contract notification17:57
th1aI'll check to make sure I did it correctly.17:58
th1aMaybe I gave it to someone else  ;-O17:58
aelknerignas: so i should load the page in question and then reload a couple of times?17:58
ignasit should do17:59
ignasthe page in question17:59
ignasis the members view for sections17:59
ignasin this case17:59
* th1a goes to take a shower.17:59
aelknerit's doing the first load now17:59
aelknerand it's taking a loooong time17:59
ignasit should17:59
ignas(on my PC 10K users in that view are taking less than a second to load)18:00
ignas(without users assigned to groups)18:00
lisppaste5aelkner pasted "learners.html profiler" at
aelknerignas: could you explain to me how to interpret the results18:03 (getSortKey)?18:05
Lumierelook in for getSortKey ;)18:06
Lumiereit should be a function18:06
aelkneruh, yeah18:07
aelknerthanks for he obvious18:07
ignasyou ran that view 3 times yes?18:11
ignasi think i got what to change18:16
ignaslet me try write the 3 liner ;)18:16
*** replaceafill has joined #schooltool18:16
ignaslisppaste5: url18:18
lisppaste5To use the lisppaste bot, visit and enter your paste.18:18
lisppaste5Ignas annotated #68868 with "The solution" at
ignasaelkner: try that18:19
aelknerignas: is _sort called by base class, and we're just overriding that?18:20
aelknerdoing three loads now...18:21
Lumierewhat's the formatter do on getSortKey?18:23
Lumiereit isn't used at all in getSortKey18:23
ignaswell - it can be used18:23
ignasand sometimes is18:23
ignasfor example18:23
lisppaste5aelkner pasted "profiler with new _sort" at
ignashmm, 3-4 times faster18:24
aelknerit would seem18:25
ignas57510 cals to getSortKey instead of 136647618:25
ignasnow remove the @profile18:26
ignasand see if you are happy with the speed18:26
ignasof the view18:26
aelkneri'll give you an informal load time in a sec18:27
aelknerpractically instant18:28
Lumierethat should be much better18:28
Lumiereshould be almost 24x faster ;)18:28
ignasLumiere: it's 4 times faster18:28
ignasthe whole view18:28
Lumiereignas: hmm :)18:28
Lumiereignas: yea18:28
aelknerignas: can we get that change into the schooltool egg?18:28
ignasaelkner: gimme a sec ;)18:29
Lumiereit just needs to be committed to the schooltool.table bzr :)18:29
ignasLumiere: there is no schooltool.table...18:29
Lumiereignas: so... still a monolithic schooltool?18:29
ignasmostly yes18:30
ignasLumiere: don't have the time18:30
ignasif everything is OK, you should get an egg soonish18:32
ignassomeone should get that fix into zc.table18:34
ignasbut eh ;)18:34
Lumieregl with that18:34
aelknerit's like your adapterTraverserPlugin directive, needed in zope!18:35
ignasnah, this one is easier to get in there, because it's 100% compatible18:35
ignasi can just commit it actually18:36
aelkneri see18:36
aelknerstill i think your adapterTraverserPlugin directive totally rocks18:36
aelkneranyway, can you send Lumiere a note when he should update his schooltool egg and how to do that?18:37
ignasbin/buildout -n ?18:38
ignasas for when18:38
ignaslet me see18:38
aelknerLumiere: you knew that already, right?18:38
LumiereI don't normally buildout -n18:39
Lumierebecause it can introduce errors18:40
aelknerit's only for these cases where you need to update eggs18:40
Lumiereignas: is it possible to buildout -n schooltool18:40
Lumiereto just update schooltool?18:40
ignasdon't think so18:40
aelknerLumiere: why would you want to?18:41
ignasto be safer18:41
ignasthe less stuff you upgrade, the less stuff can break18:41
ignasso he'd know - it's schooltool's fault for something breaking18:41
ignasinstead of having to guess18:41
Lumieresystem admins *hate* global upgrades18:42
Lumiereespecially when they are not linked to a testing system18:42
aelkneri see18:42
Lumierethose eggs are not all releases... some are svn revisions18:42
ignasLumiere: all Zope3 eggs are18:43
Lumierebut I also don't know what egg versions schooltool expects18:44
Lumiereor if there will be a release that isn't backward compat.18:44
ignasbuildbot is running functionals now18:47
ignasif they pass18:48
ignasi'll click the release button18:48
ignasLumiere: update18:51
Lumiereas I recall18:52
ignascommitted the change to zc.table too18:52
LumiereI can run buildout -n, svn up, and make18:52
ignasbut no one know how long it will take for them to release it18:52
Lumierewithout shutting down cando18:52
ignasduno about that18:52
ignasyou should be able to18:52
Lumiereok, let me send a downtime notification18:52
Lumieredwelsh aelkner:  ok taking down cando18:58
Lumiererunning buildout18:58
Lumiere(took backup)18:59
Lumierethis is gonna take a few minutes...18:59
Lumierecustom 503 warning is up ;)18:59
Lumiereschooltool 2008.10.0dev-r243719:01
Lumiereok... updated... now starting19:01
Lumiereok... done19:02
Lumierethat was faster then expected19:02
ignascool, persons view is fast19:03
ignaseveryone is happy?19:04
aelknerignas: should i remove my override of the PersonFilterWidget in cando19:04
ignaswell - you know how to profile now19:04
aelknerthe one to get rid of using group.members?19:04
ignasso - enable it, profile it19:04
ignasand see if it affects the performance19:05
*** nubae1 has joined #schooltool19:05
*** nubae has quit IRC19:06
*** nubae1 is now known as Nubae19:06
ignasok, must run now19:12
*** ignas has quit IRC19:12
*** Lumiere has quit IRC19:24
*** replaceafill has quit IRC19:56
*** replaceafill has joined #schooltool20:15
*** Nubae has left #schooltool20:28
*** jelkner has quit IRC20:37
*** jelkner has joined #schooltool20:43
*** replaceafill has quit IRC21:02
*** replaceafill has joined #schooltool21:02
*** jstraw has joined #schooltool21:13
*** jcrowley has joined #schooltool21:17
*** jcrowley has left #schooltool21:17
*** mgedmin has joined #schooltool21:19
*** elarson_ has joined #schooltool21:23
*** replaceafill has quit IRC21:52
*** rbra has joined #schooltool22:08
*** jelkner has quit IRC22:10
jstrawaelkner: ayt?22:28
jstrawI need to get a change made to the start-schooltool instance23:00
jstraw(and I got dragged away the second I ayt'd sorry)23:00
jstrawand to schooltool.paste.run23:00
jstrawand ignas isn't around23:00
jstrawI need the user and group attribute from
jstrawfor running init.d scripts as cando23:02
*** replaceafill has joined #schooltool23:09
aelkneri would guess cando:cando, but that's only a guesst23:23
*** jboisture has joined #schooltool23:57

Generated by 2.15.1 by Marius Gedminas - find it at!