IRC log of #schooltool for Thursday, 2006-07-13

srichterth1a: are you there?00:22
srichterth1a: do you know when algo arrives?00:22
tiredbonesth1a, thanks!00:22
srichterth1a: and what's the plan with driving up?00:22
mgedminI think alga planned to email you the details tomorrow morning00:25
srichterI need to plan a little bit ;-)00:26
mgedminhe has a reservation for a ticket00:26
srichteroh right, he had to wait for the visa thing00:26
mgedminbut might get a different (cheaper) one if it becomes available00:26
mgedminor something like that00:27
srichterI see00:27
mgedminanyway the ticket he kind of has had the arrival time at 15:xx on Saturday00:27
mgedminif you trust my notoriously bad memory00:27
srichterI just need to get the general jist of it, so I can see how Saturday will work out00:27
mgedmin(I don't)00:27
srichterthat sounds about right for European flights00:27
srichterok, so Saturday afternoon is good with me00:28
srichterI am glad we are not leaving before Sunday for New Hampshire00:28
srichterI have construction people at the new house all of Saturday00:28
pcarduneyou should be glad you are not driving from virginia either :/00:29
srichterhe he00:29
srichteryes, I am :-)00:29
srichterpcardune: you know, the flight to Boston from Dulles is only $12000:29
pcardunetell Jeff :)00:29
pcarduneI suppose if he pays me for traveling time... then he is losing money00:30
pcardunebut I dont think he is paying me for traveling time... oh well00:30
pcarduneat least we are fighting the good fight00:30
*** srichter has quit IRC00:35
*** pcardune has quit IRC00:44
*** pcardune has joined #schooltool01:36
*** mgedmin has quit IRC01:42
*** pcardune has quit IRC02:47
*** pcardune has joined #schooltool04:11
*** pcardune_ has joined #schooltool06:34
*** pcardune has quit IRC06:49
*** kitblake has left #schooltool08:39
*** srichter has joined #schooltool08:54
*** pcardune_ has quit IRC10:27
*** jinty has joined #schooltool13:04
*** ignas has joined #schooltool13:06
*** ZboX has joined #SchoolTool13:09
*** ZboX has left #SchoolTool13:09
*** vidasp has joined #schooltool13:09
povbot`/svn/commits: * vidas committed revision 6347:14:05
povbot`/svn/commits: Fixed broken unit tests.14:05
*** alga has joined #SchoolTool14:28
*** jinty has quit IRC14:39
*** mgedmin has joined #schooltool14:52
*** alga has quit IRC15:09
*** pcardune_ has joined #schooltool16:03
povbot`/svn/commits: * ignas committed revision 6348:16:21
povbot`/svn/commits: Make the functional test less brittle.16:21
*** pcardune_ is now known as pcardune16:22
povbot`/svn/commits: * ignas committed revision 6349:16:23
povbot`/svn/commits: Move IPasswordWriter and PasswordWriter form to just person.16:23
povbot`/svn/commits: * ignas committed revision 6350:17:08
povbot`/svn/commits: Snail food.17:08
*** jinty has joined #schooltool17:08
th1apcardune: ayt?17:08
povbot`/svn/commits: * ignas committed revision 6351:17:09
povbot`/svn/commits: 2 blank lines between top level declarations.  Make password edit view adapt person to PasswordWriter before setting the password.17:09
th1aDo you guys have a projector you can bring to NH?17:09
pcardunewe certainly have projectors, whether we can bring them is another question17:09
pcardunejeff just walked out of the room, but he should be back in a minute and I'll ask him17:09
th1aI can probably scam one if you can't.17:10
pcarduneoh, we're pretty good at scamming too17:10
povbot`/svn/commits: * pcardune committed revision 6352:17:25
povbot`/svn/commits: Creating a branch for my work on the skin.17:25
povbot`/svn/commits: * ignas committed revision 6353:17:41
povbot`/svn/commits: Add a setting for persons password changing.17:41
th1aignas:  ayt?17:54
th1aOK, I think I figured out what you should work on,17:55
th1athat is orthogonal to what we'll be doing at the sprint.17:55
ignassections ?17:55
th1aI'm worried that our memory consumption is completely out of control.17:55
th1aIs there some way we can profile that?17:56
th1aOr do we just analyze the size of objects in the ZODB and infer from that?17:56
ignasno these two correlate but are not acurate enough17:56
ignasbut it is possible to do some profiling by hand17:57
*** alga has joined #SchoolTool17:57
ignasnot as easy as speed optimization17:57
th1aRight, but it seems like the greater problem right now.17:57
ignasany examples ?17:58
povbot`/svn/commits: * ignas committed revision 6354:17:59
povbot`/svn/commits: Make browser variable names more descriptive.17:59
algasrichter: hi17:59
algaI'm arriving on SAT 15JUL at 17:0518:00
algaFlight NW 0037/ codeshare KL 603718:00
th1aignas:  What's the best way to get the memory size of a process?18:05
ignaswhat OS ?18:05
pcarduneth1a, how do you feel about WYSIWIG editors?18:06
mgedminoooh, that's a difficult question18:06
mgedminthere are all sorts of memory sizes18:06
th1amgedmin: Indeed.18:06
mgedminpaging and shared libraries complicate things a lot18:06
algaps vp $PID18:06
th1apcardune: What about them?18:06
pcarduneth1a, use KSysGuard with KDE :)18:06
mgedminor gnome-system-monitor in GNOME18:06
pcarduneth1a, we want FCKeditor in cando, should I put support for that in schooltool or leave it out?18:06
ignaspcardune: :D i need a WYSIWIG editor more than WYSIWYG one ;)18:07
pcarduneignas: ??18:07
mgedminlike SubEthaEdit or Gobby?18:07
th1a|X60s18395 pts/2    Sl+    2:02     71   815 624664 531100 51.7 /usr/bin/python bin/s18:07
th1a|X60sI guess that's not very helpful.18:07
mgedminth1a|X60s: there were headers at the top18:08
algath1a|X60s: RSS is resident set size, the actual amount of ram taken18:08
mgedminlook for, say RSS -- that's how many pages of memory are actually used18:08
th1aRSS is 53110018:08
mgedmin(but it includes things like system libraries that can be shared between many processes)18:08
alga531 megs18:08
th1aThat's after generating report cards covering two semesters at Feinstein.18:08
ignaspcardune: "What you see is what I get" (very difficult to have with IE and Firefox ;)18:08
th1aWhich touches every item in the gradebooks.18:09
th1aBut doesn't include any other data, attendance, etc.18:09
th1apcardune:  The big concern is that the code generated by a WYSIWYG editor would be messy and hard to subsequently hand-edit.18:10
ignasth1a: only data for last ~(number of threads) requests is kept in memory, so if you would go to attendance - report cards would get flushed i think18:10
alga'ps u' is perhaps more convenient18:10
algait displays RSS and VSZ (virtual memory size)18:10
algaVSZ is approximately RSS + swap18:10
algaplus mmaped files etc18:10
pcarduneth1a, for people who don't know html though... they won't be editing it with code18:11
th1aI mean messy HTML>18:11
srichteralga: hi18:12
srichterthanks for the info18:12
th1aVSZ is 625480.18:12
srichteralga: I am going to pick you up18:12
pcarduneth1a, well, are there any objects to me adding it in the skin directory?  (it could be easily disabled)18:12
th1asrichter:  Are you planning on driving to NH or going with me.18:12
th1apcardune:  Adding what?18:12
algasrichter: yay, thanks!18:13
pcarduneth1a: FCKeditor18:13
th1aMaybe I don't know what you're talking about.18:13
pcardunei meant to say objections instead of objects (woops)18:13
th1apcardune:  Oh, I wasn't following you at all.18:13
th1aIs that for entering comments or something?18:14
pcardunefor entering anything that is html18:14
th1aFor users to enter things.18:15
srichterth1a: well, we could all go up together18:15
th1aWhy not Kupu?18:15
pcardunelike journal entries, or curriculum18:15
srichterth1a: Otherwise I Would need to rent a car18:15
th1aThen I will pick you up.18:15
pcardunewhy kupu and not FCKeditor?18:15
pcardunewhat is your preference?18:15
th1aWell, I should look at FCKeditor.18:15
th1aKupu is just more common in the Zope world.18:16
mgedmin"This is an object-oriented system.  If we change anything, the users object."18:16
algath1a: my airfare is approx $1600.18:16
th1aalga:  Whew!18:16
* pcardune winces18:16
algaand that's business class on my way there!18:16
algaI'll have laptop power! :-)18:17
th1asrichter:  Did you hear about the tunnel?18:18
th1aalga:  A big section of the tunnel running to the airport fell on someone this week.18:18
th1aLike a 10 by 12 meter section fell.18:19
th1aOn someone's car.18:19
srichterth1a: We might take the back route18:19
th1aI imagine it is still closed.18:19
srichterth1a: kupu sucks ;-)18:19
srichterth1a: In Zope 3 land tinyMCE and FCKeditor have been integrated before18:20
alga<ignas> bye, Albert18:20
pcardunetake that th1a!... I overheard some good things about tinyMCE18:20
mgedminwysiwyg is for wimps!18:21
th1aOK... I yield.18:21
* mgedmin wants vi embedded into a browser18:21
mgedminnow *that* would reduce comment spam18:21
th1aI want Logo embedded into my browser:  http://logowiki.net18:21
mgedminAlan Kay's EuroPython keynote was done with Squeak embedded in a browser18:22
mgedminvery impressive18:22
th1aYes, when I saw him present Squeak a few years ago it rearranged my brain cells.18:23
th1aUnfortunately, it is hard to replicate that experience on your own with Squeak.18:24
pcarduneth1a, jeff says we can bring a projector18:24
th1apcardune: Excellent.18:24
mgedminnow all you need is popcorn and some DVDs18:25
th1aignas:  My memory consumption isn't going down.18:25
ignasth1a: is it going up ?18:25
th1aI'm already bringing the beer.18:25
th1aignas: No.18:25
pcarduneif srichter is still in germany.... bring us back some good beer :) (tom's paying)18:26
ignasth1a: python is not reducing it's memory consuption even if it's using less memory in reality18:26
mgedminoh right, you don't have any good beer on that side of the pond, do you?18:26
mgedminactually, I think "good beer" is an oxymoron18:26
* mgedmin does not like beer18:26
th1aI haven't had beer in Germany yet... I do like English pub ale.18:27
*** jelkner has joined #schooltool18:27
algaMmmm, I had a chance to try English pub ale in May18:27
algathat was something18:27
th1aActually, it is hard finding good beer in Vilnius.18:28
pcardunemgedmin: do they have absynthe in Vilnius?18:28
mgedminno idea18:28
pcardunethat would make for an interesting sprint18:28
th1aThe Wormwood Sprint.18:28
th1apcardune: Is that what the hip college students are drinking?18:29
pcarduneno... they aren't nearly that classy.  Absynthe was legalized in my canton in switzerland about 5 months before i left18:29
th1aignas:  I was also worried because starting up five ST instances on our new testing server took up 800 of our 1500 megabytes of RAM.18:30
th1aI guess that's not entirely shocking,18:31
ignasth1a: starting 5 instances of Zope3 would do the same i think18:31
ignasth1a: i would think that having Schooltool as a content object would help :D18:31
ignas1 Zope3 instance 5 schooltools ;)18:31
th1aFor multiple schools, absolutely.18:31
ignasData.fs would be humongous though :/18:32
th1aWe'll have to get back to that at some point.18:32
th1aignas:  So do you not think this is a pressing problem?18:33
th1aIt is a little hard to tell without profiling.18:33
ignasi think that it might be a problem. Real profiling memory usage is something i haven't done ever :/ Only adhoc click click, ps a | grep18:35
th1aI guess the Data.fs isnt' actually that big.18:35
povbot`/svn/commits: * pcardune committed revision 6355:18:35
povbot`/svn/commits: Made an actions bar that replaces both navigation and action menus from before.18:35
th1a111 Mb.18:35
th1aWhich is kind of weird actually...18:36
th1aHow is the memory consumption so much bigger than the db size?18:36
th1aPerhaps I don't understand how these things work at all.18:36
algawell, for one, each char in a unicode string takes 2 or 4 bytes18:36
ignasth1a: serialized objects are smaller than objects in memory18:36
algain memory18:37
algaand if it's all ascii, then 1 char in Data.fs18:37
algaalso, there are 4 threads caching the same data18:37
ignasth1a, what i was saying that having 5 schools each with sample data as content objects would get the data.fs into huge numbers18:37
th1aignas:  Yes.18:38
th1asrichter: Are you still planning on calling me?18:38
ignaswith separate schooltools we have 5 Data.fs files of 111 Mb, and 5 proceses of ~150 Mb. With schooltool content objects we'd have something like 1 Data.fs of 555Mb, and 1 Zope3 instance of 180 Mb ...18:39
ignasnumbers might be a bit different in real life ;)18:40
jintyin some ways it's nice to have the servers in different instances because it makes migration to different servers (if necessary) easier18:40
th1aYes, if we supported ST content objects, that'd increase the need for some kind of XML export.18:42
ignasth1a: the problem is that we  don't have any metrics for speed/memory performance, as well as we don't have any targets.18:43
* jinty tries and fails to find the knob for changing the default number of app threads18:43
th1aignas:  That was my original point... I've got a feeling that ST is turning into a memory pig, but I'm not sure.18:43
th1aSo if we could profile it fairly easily, that would be good... but I guess it isn't easy.18:45
ignasprofiling will not help without knowing "how small/big do we want our pig to be"18:45
mgedminjinty: put something into zope.conf18:45
jelknerth1a: tom, are you here?18:46
th1ajelkner: Yes.18:46
mgedminjinty: e.g. 'threads 4'18:46
mgedmin4 is the default, I think18:46
th1aignas: Well, I guess I was thinking that half a gig was a good target.18:46
jintymgedmin: that also works in schooltool?18:46
mgedminum... dunno18:47
jintymgedmin: I thought you had shorted out zope.conf18:47
mgedminyeah, I think so too18:47
jelknerth1a: did you see my message?18:47
mgedminI see 'thread-pool-size' in schooltool.conf.in18:47
mgedminit definitely worked in the old twisted-based schooltool18:48
mgedminI do not know if it is hooked up nowadays18:48
th1ajelkner:  Not sure.18:48
ignasth1a: with how many concurent users? how big the school is ?18:48
th1aOh... I see it.18:48
mgedminbut I'm sure it wouldn't be too difficult to hook it up18:48
jintymgedmin: yeah, but I think those are the front end twisted threads18:48
jintymgedmin: or something like that18:48
* jinty gets giddy when it comes to twisted18:48
mgedminsame thing, really18:49
mgedminN threads that process requests, and each has its own in-memory ZODB object cache18:49
jintyjust something I can change in an emergency if the testing server starts swapping18:50
th1aignas:  OK... guess I'll keep thinking about this.18:56
ignasth1a: i think we should either start measuring performance, memory consumption seriously or wait for our users to report unusable views and only fix them then18:56
ignasth1a: not do some random fixing when we feel like it :/18:57
th1aI'm just talking about doing measurement at this point.18:57
th1aAlso, there are a number of different deployment stories in play.18:58
th1aWe would like SchoolTool to run on random Edubuntu installations, for example,18:58
jintyahh, looks like it.18:59
th1abut in reality, at this point it may require its own somewhat beefy server.18:59
ignasth1a: i can start thinking of a framework for measuring schooltool memory consumption19:01
jintyer actually, it looks like thread-pool-size is actually hooked up, but with a default of 1019:02
mgedminwhoa, 1019:03
jintyyeah, seems excessive;)19:04
povbot`/svn/commits: * ignas committed revision 6356:19:04
povbot`/svn/commits: Hide edit password menu item when user should not see it.19:04
mgedminwe had a zope 3 app that ate 600 megs of ram19:20
mgedminchanging the number of threads from 4 to 6 brought the server on its knees, swapping19:20
th1a|X60sNot that I want to change this, but would using a relational database theoretically use less memory than the ZODB?19:21
ignasth1a|X60s: maybe, i'd say it would cause a lot of design decisions to change, geting out 200MB of data out of relational database every request is not a good practice, thus the application would be structured differently19:23
mgedminI think it depends on the caching strategy19:23
mgedminbut then you would have the DB server eating its own portion of RAM19:24
mgedminwith its own caches19:24
mgedminunless you move the DB server onto a different machine19:24
jintyperhaps ZEO19:26
ignasIMVHO ZEO will not decrease memory consumption19:27
jintybut the memory will be on a different machine?19:28
ignasthe way Zope3 interacts with it's data storage backend is what is eating the memory19:28
ignasZope3 will still load it into the process that is doing the calculation19:28
ignasand would load same persistent object instances19:28
ignasi might be wrong though ...19:28
jintyyou probably have a better idea than I do19:29
*** dom has joined #schooltool19:45
domis there a plugin interface with schoolbell/tool?  is it possible to add additional fields to the events without modifying schoolbell itself?19:49
*** ignas has quit IRC19:49
th1adom:  Hm... that's a complicated question.20:03
*** vidasp has quit IRC20:03
domI figured it might be.20:03
th1aI mean, SchoolTool is very exensible,20:03
th1aso creating new objects is pretty easy.20:03
th1aBut I'm not sure what the best angle for modifying the event object is.20:04
th1aI guess you'd use an adapter.20:04
th1aWhich is not difficult once you understand the abstraction.20:04
th1aWhat do you want to add?20:05
domsry, just a sec. loading the two pages I need for comparison20:08
domokay, so an event has title, description, location and then time/recurrance info, right?20:08
domwe want to track additional information since our resources are being booked by professors for seminars or conferences20:09
th1aYou can already book resources as well.20:09
domso the additional fields we are trying to track are speaker, affliation, "event URL" (meaning an external link),  contact info (e.g. who to call), and aproximate attendance size20:10
domso a handful of additional strings, a url, and an int.  roughly20:11
th1aNoting that I'm the project manager and not one of the main developers...20:12
th1aI think the general answer is that you'd want to use a Zope 3 "annotation" for the event to do this.20:13
th1aOf course, the problem is that you probably have to learn more Zope 3 than you want to to get to that point.20:14
th1aA fact which causes me great consternation.20:14
domThat is sortof what I was afraid of.  We can't really get into Zope dev to patch schoolbell to get this feature20:15
dombut schoolbell is the best match for our needs so far20:15
th1aWell, the nice thing would be that you could do it in a way that was cleanly separated from the base SchoolTool code.20:16
th1aMost of our developers (in Lithuania) are not here anymore.20:17
th1aI mean, they've gone home.20:17
th1aBut they can give you a more precise answer.20:17
jelknersrichter: stephan, are you here?20:17
th1aOr you could send an email to schooltool-dev.20:17
domokay.  maybe I'll post on the list and see what people think20:17
th1aOK.  I have to run.  Good luck dom.20:18
domthx tom20:18
jelknerth1a: tom, did you get my email?20:19
*** jelkner has quit IRC20:43
*** jinty has quit IRC21:12
pcardunesrichter or mgedmin, I'm in doctest hell!21:25
pcardune    - <a href="http://localhost/++etc++site/default/RootErrorReportingUtility/@@configure.html">Configure</a>21:26
pcardune    +         <a href="http://localhost/++etc++site/default/RootErrorReportingUtility/@@configure.html">Configure</a>21:26
pcardune    ? ++++++++21:26
pcardune    - ...21:26
pcardune    - <a href="http://localhost/++etc++site/default/RootErrorReportingUtility/@@index.html">Error List</a>21:26
pcardune    +         <a href="http://localhost/++etc++site/default/RootErrorReportingUtility/@@index.html">Error List</a>21:26
pcardune    ? ++++++++21:26
pcardunein devmode/devmode.txt21:26
* mgedmin suggests schooltool.pastebin.com21:27
pcarduneI didn't know there was a pastebin21:27
mgedminwell, it is horribly slow today21:27 then?21:27
mgedminanyway it looks like your problem is caused by a menu item or a separator that was there before these links but now disappeared21:30
mgedminif you expect 'line1\n...\nline2\n', but get 'line1\nline2\n', you'll get a failure21:30
mgedminI consider it a bug in zope.testing.doctest.py21:31
pcarduneno kidding21:31
mgedminwell, not clash, but you get what I'm saying21:31
pcarduneso, how would you suggest getting around this?21:31
tiredbonescan someone give me a pointer to this upcoming sprint. Time and place would be nice.21:42
povbot`/svn/commits: * pcardune committed revision 6357:22:03
povbot`/svn/commits: fixed broken functional tests22:03
povbot`/svn/commits: * pcardune committed revision 6358:22:06
povbot`/svn/commits: adding FCKeditor resources22:06
*** alga has quit IRC22:24
*** mgedmin has quit IRC22:25
*** tiredbones has quit IRC22:27
*** tiredbones has joined #schooltool22:27
*** mgedmin has joined #schooltool22:53

Generated by 2.15.1 by Marius Gedminas - find it at!