IRC log of #schooltool for Monday, 2012-11-05

*** ignas has quit IRC01:24
*** yvl has joined #schooltool09:32
*** khildin has joined #schooltool10:13
*** paulproteus has quit IRC10:31
*** paulproteus has joined #schooltool10:34
*** khildin_ has joined #schooltool10:54
*** khildin has quit IRC10:55
*** ignas has joined #schooltool13:42
*** replaceafill has joined #schooltool15:07
*** menesis has joined #schooltool16:29
th1ahi menesis, replaceafill, aelkner, yvl.16:31
replaceafillgood morning/afternoon16:31
yvlgood morning16:31
th1aI just sent a bug from Glenda to menesis & yvl.16:32
th1aSo... that's the first order of business.16:32
replaceafilli have a bug i'd like to discuss too16:32
replaceafillwell, i think it's a bug16:32
replaceafillnot sure :)16:32
th1areplaceafill?  Theories, next steps?16:32
th1aIs this related?16:32
replaceafillth1a, ah, mine is not related16:32
replaceafillor maybe...16:32
replaceafilli'm just confused16:32
replaceafillcould you guys see that?16:33
replaceafillwhile working on jelkner's instance16:33
replaceafilli reimported his skills16:33
replaceafillbecause he had some typos16:33
replaceafilland schooltool.quiz has this report16:33
replaceafillthat relies on equivalent skills16:33
replaceafill(because jelkner's sections are multi-term)16:34
replaceafillbut that ^ part of the importer16:34
replaceafillremoves equivalences16:34
replaceafill(if i understand it correctly)16:34
replaceafilland i don't understand why16:34
replaceafillthis could affect the projects gradebook btw16:34
replaceafillsince it also relies in equivalences16:35
* replaceafill done with his "bug" report16:35
yvlhmm, ok16:36
yvlso first, the bug th1a sent16:36
yvlI know one case where similar thing may happen by design16:36
yvlyou might remember scores from different teachers (and different sections IIRC) ending up in some gradebooks16:37
yvlbut really, I have no idea what is happening there16:39
yvlData.fs would help a lot16:39
yvlbut we probably can't ask for it?16:39
th1aThat's not our first move.16:40
th1aIf we can avoid it.16:40
yvlI can stare at evolution and try to come up with scenario that broke grades16:40
th1aI bet it is similar to jelkner's problem -- equivalent standards.16:40
aelknerequivalent skills is an issue on the import16:41
th1aWell, we can ask them anything... or do a chat.16:41
replaceafilli have the same feeling about it16:41
aelknerin that if the user doesn't fill in those cells, they will be wiped out16:41
th1aOr is there some kind of script or view that would dump useful info to us.16:41
replaceafillaelkner, shouldn't those be wiped out "if" there is data in the cells?16:42
th1aI'm not saying this is a reimport issue for Glenda's bug.16:42
th1aI'm just saying, that eqivalencies seems a likely suspect.16:42
aelknerreplaceafill, we could change the logic for that column if we wanted to16:43
aelknerit would be weird though because it would not work like most import cells do16:43
th1aCould we stick on Glenda's bug at the moment, please.16:44
aelkneri mean, if a person last name is blank, but it is not blank in the data, do we blank it out?16:44
th1aCan we write a script or view that will dump out useful information to us?16:45
aelknerwe have an export feature16:45
yvlno, aelkner16:45
yvlwith some effort yes, th1a16:45
th1aWell, we need some kind of plan.16:46
yvlwe need to think about it firs16:46
yvlso you don't want us to ask for the data, unless no other option16:47
th1aWe need to try not to rely on that.16:47
yvlhow much time are you willing to assign to resolve this bug?16:47
yvlalso - how much dev time?16:47
th1aIt isn't so much a matter of how much dev time as how quick can we get it done.16:48
yvlI understand16:48
yvlfastest - get me an uncompressed Data.fs16:48
th1aAssuming this guy can download and run a script.16:48
yvlfiguring out what to search for is somewhat... trickier16:49
yvl"a script" is a thing that does what I would do, but automatically16:49
yvland does not show the grades16:49
th1aYes, but also it is just a way of giving us anonomyzed data.16:49
yvland what I would do depends on what I would see16:50
th1aThe main thing is stripping out personal data.16:50
yvlok, I'll think about this16:50
yvlit's not that I want to see data16:50
yvlI want to see what happened to data16:50
yvlbut ok16:51
yvlI'll come up with something16:51
th1aI'd prefer to get something to them today.16:51
th1aIf replaceafill wants to take a crack at it.16:52
replaceafilli wouldn't know where to start :(16:52
th1aOK.  So we'll shoot for something tomorrow morning.16:52
yvlI'll see what I can do16:53
yvlblind debugging is not trivial sometimes16:53
yvlwell, at least I have a hunch where to look now :D16:54
yvlseveral, actually16:54
th1aI can say "We'll have a script that'll pull some anomymized data tomorrow or just send us your Data.fs today.16:55
yvlyou can16:55
th1aOr, literally, maybe we should just write something that copies a Data.fs and blanks out names and demographics.16:56
th1aHow hard would that be?16:56
yvlto seem like it was blanked out - easy16:56
yvlbut ZODB is append-only DB16:57
th1aPack it?16:57
yvlwe'll loose history, and history is what I need16:57
yvlmost likely it's not going to be a script that pulls data16:57
yvlit'll rummage through history of data16:58
th1aSo ideally we'd have a zipped but not packed Data.fs from them, right?16:59
yvlso, if I don't get one by tomorrow morning my time, I'll start working on a script17:00
yvlsounds good?17:00
yvlwe'll get this sorted, th1a17:00
th1aI know.17:00
yvlIf they are sending the Data.fs17:00
th1aI get frustrated when I feel like things would be easier if they could just dump a few tables from MySQL.17:01
yvlI would ask to also send the backup they made Thu17:01
* yvl too th1a, yvl too17:01
yvlbtw, do you have an idea on how many students we are talking here?17:03
yvlorder of magnitude17:03
th1aOh... no.17:03
th1aOK, moving on.17:04
th1aShifting back to replaceafill's bug?17:04
replaceafilli just wonder how would it affect projects17:05
replaceafilli'm setting up an instance to see17:05
th1aSo do we need to repair those?17:06
th1aAnd make the importer not remove the equivalencies?17:06
replaceafillth1a, probably17:06
yvlI think that part is a bug17:06
yvlbecause it kills all equivalencies17:06
replaceafilli'm worried about multi-term sections17:06
replaceafilllike in jelkner's case17:07
th1aIt sounds like a bug to me.17:07
replaceafillthey rely on equivalences17:07
th1areplaceafill is too gentle sometimes.17:07
aelknerok, here's the thing about not removing equivalencies:17:07
aelknerif the data has equivalency a and the import a, b, what is the logic?17:07
aelkneras in all other cells of that type, the logic is currently:17:08
aelknerremove a, add a an b17:08
yvlyes, I can see the code17:08
aelknerwhat should the logic be?17:08
yvlit should check for base level container17:09
yvlyou "add" equivalency only for global skills17:09
yvlyou should remove also for global skills only17:09
aelknerit we consider the purpose of the field to be only for adding global equivalencies, then that would make sense17:10
yvlis it possible in any way to add not global equivalencies?17:10
aelknerok, sorry, i'm not even sure what an equivalency is that is not global17:11
aelkneri thought all nodes are found in the global container17:11
yvlnodes - yes17:12
yvlskills - no17:12
yvlprojects also contain skills17:12
replaceafillyou can have skills in projects17:12
aelkneroh yes, sorry, we're talking about skills, not nodes17:13
replaceafilla bigger question from mine17:13
replaceafillis it possible to recover? :(17:13
replaceafilljelkner's instance lost all equivalencies17:13
replaceafilland i can see him using projects in the near future17:14
aelknerreplaceafill, does jelkner have a backup?17:15
replaceafillaelkner, yes17:15
aelknerand the problem started after doing an import?17:16
replaceafillas i said, he had some typos in his skills17:16
replaceafillso we reimported only the Skills sheet17:16
aelknerok, i'd say fix the import to not remove equivalencies if they are not global skills17:16
aelknerrevert to backup and reimport17:17
replaceafillwell, he already has new grades in17:17
aelknerhe would have to reenter them17:17
yvlyou know what17:17
yvlyou can run with old database17:17
yvlfigure out what project skills had equivs in global skills17:18
yvlmake a list17:18
yvlrun with new database17:18
yvlstitch them again17:18
th1aI'd note that while it SHOULD work, reimporting to fix typos is something I'd not do.17:18
th1aUnless you're trying to find bugs.  ;-)17:19
yvlreplaceafill, did that make sense?17:19
replaceafilli didn't expect it at all :(17:19
replaceafillyvl, yes17:19
replaceafilli was thinking of a view to spit those17:19
yvlyou can just dump the list into a .py file17:19
yvlin a debug method17:20
replaceafilli have already the two Data.fs in place17:20 files are text files and it's easy to write stuff like17:20
yvlfix = [17:20
yvl('id1', 'id2', 'id3'),17:21
yvland then just import the file :)17:21
replaceafilli was using prints  + copy/paste :D17:21
replaceafillthe .py idea is better ;)17:21
yvljust one of the things I like about languages like Python17:22
replaceafilli'm done with jelkner's bug17:24
replaceafilli mean, questions about it17:24
th1aSo... menesis?17:25
th1aWe'll let you go.17:25
th1aIt looks like we'll need a CanDo bugfix this week.17:25
th1aok yvl, anything else to report from your holiday?17:28
yvlI had a nice time :D17:28
yvlbut seriously, no.17:28
yvltinkered with reports, that's all17:28
yvlthanks for the list by the way!17:29
th1aYou're welcome.17:29
* yvl claims section roster and calendar :)17:29
th1aI have no particular inspiration for the calendar ones.17:29
th1aSo if you have ideas, go for it.17:29
th1aA lot of them should be easy layout-wise, if we're doing this right.17:30
th1aBasically we need a robust big grid widget.17:30
th1aThat knows what to do if if overflows the pages.17:30
yvland that does not crash with a traceback in reportlab :D17:31
yvlthere's been a nasty bug there in extremly nasty part of reportlab's code (called platypus, figures)17:32
yvlok, /me done17:32
th1aWell, also, we might just need to be aggressive about pre-processing to sort out the data before sending it to ReportLab.17:33
th1aOK, thanks yvl.17:34
aelknerok, so i made significant progress with the aggregate search feature, but i have some questions17:34
aelknerfirst, log in as manager for this view:17:35
aelknerbtw, the skills document in this case still has the old extra node levels17:35
th1abtw, this gives me a traceback:
aelknerah, yes, that, we need to fix that bug as a separate task17:36
aelknertry just hitting the Cando tab and logging in17:36
aelknerthen go to the add skill link17:36
yvljust log in and open the same page again17:36
th1aYes, I'm there.17:37
yvlnice progress, aelkner17:37
aelknerah, thanks yvl17:37
aelknerbtw, we can't filter sklilsets and skills by level because the index can't have that value in it17:37
aelknerso that's why the bottom two 'layers' are really skillsets and skills17:38
aelknerregardless of which layer they may be in17:38
aelknerthere's just no way around that because we can't have a layer field in the index for skillsets or skills17:39
yvlactually it's because we decided to add two bottom layers that do not contain nodes but are used to "rename" skillsets and skills17:39
th1aThat has no surprising implications for me as the user, right?17:40
aelknerit just means that search for skillsets or skills can't be sub-filtered by layer17:40
aelknerok, so moving on from that, there is another issue about the nodes filtering17:41
th1aI have no idea what that means aelkner, but as long as the form is not lying to me, I don't care.17:41
aelkneryou have no idea what that means, what is that?17:42
th1a search for skillsets or skills can't be sub-filtered by layer17:43
yvlaelkner probably meant "sorted"?17:43
aelknerwell, skillsets could have two different layers, right?17:43
aelknerif they create two documents with two different layers for skillsets17:44
yvlah, right17:44
th1aYes, and I can see that I can't sort by just one of them.17:44
th1aThat's what you're saying?17:44
aelknerthen the search would yield skillsets that have different values in the layers column17:44
aelknerwhat i was asying is: we can't allow the user to filter out skillsets only for a certain layer17:44
th1aOK.  Got it.17:44
th1aMoving on...17:44
aelknerayway, i didn't thinkl that would be a major problem17:45
aelknerjust wanted to mention it17:45
aelknerok, ack to the nodes filtering17:45
aelkneri used the 'any_of' logic in the layers index we discussed last week, and it worked nicely17:46
aelknerone thing tough, what if a node has no layer?  do we show it in the search17:46
aelknerand how?17:46
aelknerfirst, do we show it?17:46
aelknerit's a question of the meaning of the checkboxes17:47
aelknerthey say: show nodes for the layers that are checked17:47
aelknerso what if a node has no layer?17:48
yvla good question17:48
aelknerif nodes are added in the document views, they would never be added without a layer17:49
th1aA "No layer assigned" checkbox might be easiest.17:49
th1aUnless it is hard.  ;-)17:49
aelknerth1a, that would answer the question of the user interface17:49
aelknerthe next question would be: how do we get those nodes via the index?17:50
aelkneryvl, do you have an idea how we would do that?17:50
yvl{'none': catalog.extent} ?17:52
yvlindex.apply({'none': catalog.extent}17:52
yvlThe 'none' argument takes an extent and returns the ids in the extent17:53
yvlthat are not indexed; it is intended to be used to return docids that have17:53
yvlno (or empty) values.17:53
yvl[from documentation]17:53
aelknerthing is, nodes that have no layer are still indexed by the 'text' index17:54
yvlbut are they indexed by a layer index?17:54
yvlroughly speaking, catalog.extent includes all object intids that were indexed by catalog's indexes17:55
aelknerright, so all of them basically17:56
yvlso there you probably want to ask - what nodes are in catalog.extent that are not in layers index17:56
yvlor layer_names17:56
yvlwell, the other one, not the one that indexes titles17:57
aelknerah, set logic between the two results17:57
aelknerok, i'll look into that17:57
aelknerth1a, where would you want the 'No layer assigned' checkbox, below Skill or above Skillset?17:57
th1aAt the bottom.17:58
aelkneryvl, tests are broken partly because the indeces are not set up in tests17:58
aelkneri found that if i changed the filter routine to do a catalog.reIndex, i updated the index to have the layers17:59
aelknerbut the test infrastructure should probably have something to make that not necessary17:59
yvlit may be just not enabled for some of the test layers18:00
yvlwhich ones were failing?18:00
aelknerthe 'layers' index for the nodes was empty18:00
yvlok, I see there are more broken tests atm18:01
yvlwill need to fix18:01
aelknerwell, partly the import_skills.txt broke because skillsets and skills are included in the search results now18:02
aelknerthat is not in itself a problem18:02
aelkneri was going to change the tests to uncheck the skillsets/skills checkboxes to recreate the18:03
aelknernodes only search that the test called for18:03
aelknerthe problem then came up with the empty index18:03
aelknerand also the issue of nodes with no layers which we just addressed18:03
yvlso if you can carefully add the reindex string and fix the tests18:04
yvlthen remove the reindexing and commit those somewhere18:04
yvlit would be easiest for me to fix18:04
* yvl would be grateful18:05
aelknerah, ok, so you're saying add the reindex to the filter method that i mentioned earlier18:05
aelknerthen fix the tests, then remove it and email you>18:05
yvlumm, no18:05
yvladd the reindex hack, fix the tests, remove the hack, keep tests broken18:06
yvland when you push to your lp branch I can pull and fix from there18:06
aelknerisn't that what i just said?18:06
yvlsorry, I misread18:06
aelknerte reindex hack is in the filter method18:06
yvlthought you wanted to email me a diff18:06
aelkneroh, no, just the fact that my branch had the hack and test fixes that you could then look at18:07
aelknerok, we're on the same page there18:07
yvlyes :)18:07
aelknerok, next18:07
aelknerdo we want a clear search button that would clear the text filter and put the checkboxes back to all checked?18:08
aelkneresentially, the same as calling up the view for the first time18:08
th1aIt doesn't seem particularly necessary to me.18:08
aelknerth1a, that question is for you18:09
aelknernever mind that then18:09
th1aI would note that there is no real reason for the form to display results at all when you call it up.18:09
aelknerooh, that's a good point18:09
aelknerbecause it takes the longest when you first call it up because no filter has been put into placve18:10
aelknerth1a, would you like the result to be empty on initial call up of form?18:10
aelkneri like that idea18:10
aelknerthat's a real improvement i think18:11
aelknerok, the last thing that is on my list is changing the search when the user is looking only for nodes18:13
khildin_Q: Anyone of you understands Dutch?18:13
aelkneras in, in the case where the user is editing the node children18:13
aelknerin that case, the skillsets and skills are not relavent18:13
yvlkhildin_, nope18:13
aelknerso i need to change that table to not show the Skill Sets and Skills checkboxes18:13
aelkneranyway, that's it for me18:14
th1aOK.  Thanks aelkner.18:14
th1amenesis, ayt?18:14
aelkneryvl, i'll email you when i've updated my branch to have the reindex hack18:14
menesisth1a: yes18:14
yvlsure, thanks aelkner18:14
th1aAnything to add?18:15
th1aI know you were on holiday.18:15
khildin_yvl: I ask because I'd like to discuss a requirement for Dutch schools to extract information from a school administration tool to give to the authorities... if schooltool can generate that (it's cvs) schooltool might get a much bigger chance to get implemented in the Netherlands...18:15
menesisone day only18:15
menesisI have no idea about the cando bug you sent18:16
th1aYou mainly need to know that we'll hopefully be pushing a fix this week.18:16
th1aAlong with some others.18:17
menesisemail me when fixes are done18:17
menesisas for me18:17
th1aYes... replaceafill did you do all the recent CanDo fixes?18:17
replaceafillth1a, yes18:17
th1aMake sure menesis knows what is what.18:18
replaceafilli fixed the bug that prevented deleting sections18:18
replaceafillit's in trunk now18:18
replaceafillmenesis, it's in rev 134 schooltool.cando18:18
menesisso maybe I can push those fixes already18:19
replaceafillmenesis, and rev 133 has the fix for validation in the score student view18:19
menesisth1a: I have played with juju18:21
th1areplaceafill did too.18:21
menesiscan't say "worked on" because even the samples don't work18:21
th1aYes, that was my experience.18:22
replaceafillit worked for me18:22
replaceafilli wasn't able to access wordpress using a browser though :D18:22
replaceafillbut i did "wget http://..." and saw it ;)18:22
menesiseven after third try when I saw units being installed18:22
replaceafill(it's a problem when you're using a vm inside another vm)18:23
th1aDid it say pending replaceafill?18:23
menesisthere was an error on mysql because InnoDB engine is not installed or configured or something; can't find the log now18:23
replaceafillth1a, at some point, yes18:23
th1aWell, yes.18:23
replaceafillth1a, you mean "juju status" right?18:23
th1aDid it ever say running?18:23
replaceafilland then changes to "started" iirc18:24
replaceafilland gives you a public ip18:24
replaceafillfor the service18:24
replaceafill(or whatever is called)18:24
th1aOK, so there is a mysql problem?18:24
* replaceafill doesn't know the terminology yet18:24
replaceafillnot for me18:24
replaceafillmysql also started18:24
replaceafilli didn't connect to it though18:25
replaceafilllike using the command line18:25
th1aIt must have started for wordpress to work.18:25
replaceafillthat's what i though18:25
replaceafilland i found why there's no apache charm18:25
replaceafillwordpress seems to use ngnix18:26
replaceafillnot apache18:26
replaceafillat least i didnt see an apache process18:26
replaceafilli did notice ngnix bla bla18:26
menesisyes, wordpress is configured to use nginx18:26
menesisI tried wordpress/postgresql, but "there are no matching endpoints". tried python-django/postgresql, but django failed to install, too18:27
th1aOK, so at this point, replaceafill got it working but doesn't remember how?18:27
replaceafilli tried minecraft :D18:27
menesisI have dokuwiki working18:27
menesisno dependiencies18:27
th1aSo really just the wordpress demo is broken.18:27
menesisor rather, relations18:27
menesisit installs apache18:28
menesisand php, and whatnot18:28
menesisthere is no need for standalone apache charm18:28
th1aHm.  OK.  Charms can just have several packages.18:29
menesisnow I am going to write a simple charm to install schooltool18:29
menesisas it is18:29
replaceafillmenesis, is it really necessary to destroy your environment after rebooting?18:29
replaceafillbut we can talk about it later :)18:30
menesisreplaceafill: not sure18:30
replaceafillit's a "limitation" according to the docs18:30
replaceafillof local envs18:30
menesisI read „Local provider environments do not survive reboots of the host at this time, the environment will need to be destroyed and recreated after a reboot“ at
menesisas I understand this applies to lxc way18:32
menesisI think there should be a way to create a cloud of virtual machines18:33
menesisor some other private cloud18:34
menesisbut haven't read about that yet18:34
th1aOn Amazon, the processing and storage are completely separate, right?18:35
menesisit should be18:36
menesisEC2 is "processing" part and S3 is "storage" part18:37
menesisfor initial experimentation LXC should be enough for me18:37
th1aThanks menesis.18:39
th1aJeez... I think replaceafill still has to go...18:39
replaceafilland i have questions!18:39
replaceafillbut they're mostly for you th1a18:40
th1aGo ahead.18:40
replaceafillso as i said i fixed the delete section bug18:40
replaceafilland kept working on iep's18:40
replaceafillfirst question:18:40
replaceafillshould teachers be able to see student's iep skills?18:40
replaceafilli mean in the student's iep page18:41
th1aDo they now?18:41
replaceafillin the old implementation everything was protected by schooltool.edit18:41
th1aYou should ask Welsh, I guess.18:41
replaceafillso only managers were able to see the iep page18:41
th1aThat's not what I'd think but it is his deal.18:41
replaceafillteachers were able to see a different color for the skill in the student's gradebook18:42
th1aOh, go ahead and let them see it and see if he complains.18:42
replaceafillah ok18:42
replaceafillsecond question18:42
replaceafilli have a UI problem18:42
replaceafillbecause, again, in the old implementation18:42
replaceafillyou know18:43
replaceafilllog in as manager:cando18:43
replaceafillsee the Sections part18:43
replaceafillin the page18:43
replaceafillbtw, the iep link is only available for "learners" in the person index page18:43
replaceafillso, my question18:44
replaceafillclick on:18:44
replaceafill2010-2011 -- Welsh, David -- Ancient Observatories18:44
replaceafillthat's a edit view18:44
replaceafillthere's really not "view" view for iep skills18:44
th1aWell, it should be consistent with the rest of SchoolTool now.18:45
replaceafillso i'll insert a "view" view in the middle18:46
replaceafilli'm also in the middle of changing the score student view to show the iep color18:46
replaceafillthen comes the three virginia reports18:46
replaceafillthat needs fixing18:46
th1aAh, yes.18:47
replaceafilli think that's it from me18:47
replaceafilli'll jump to fix jelkner's data.fs18:47
replaceafillyou know jelkner ;)18:48
th1aYes, do that first.18:48
* replaceafill done18:48
th1aAlso, you could just roll him back, couldn't you?18:48
replaceafillwe just talk and we're:18:48
replaceafillgoing to roll back18:48
replaceafilltest that everthing works for his demo tomorrow18:49
th1aI just sent the email about the VA bug.18:49
th1aOK, thanks guys!18:50
th1aWe're in that post-release cleanup malaise.18:50
th1aHopefully we'll be through it soon...18:50
th1aSee you Wednesday.18:50
th1aGood luck!18:50
* th1a drops the bag of gravel.18:50
yvlthanks guys, see you soon18:50
replaceafillthanks guys18:50
replaceafillsee you next monday :D18:50
replaceafilli'll be traveling on wed ;)18:50
aelknerthanks guys, cya wed18:51
aelknerreplaceafill, cya monday18:51
*** yvl has quit IRC18:51
replaceafillth1a, jelkner's instance data.fs restored19:16
replaceafillhe just lost today's quiz answers :(19:16
replaceafillbut he's fine with that19:16
replaceafillhe'll re-give the quiz next week19:17
*** menesis has quit IRC19:22
replaceafillah, now i understand some of "juju status" output :D19:28
*** ignas has quit IRC19:49
*** menesis has joined #schooltool20:05
th1areplaceafill:  I just sent a link to the broken Data.fs.20:34
th1aIf you've fixed jelkner's bug may be you can take a peek.20:35
replaceafillth1a, i restored jelkner's data.fs, i'm now fixing the bug in the importer21:39
replaceafillto work only on global skills21:39
*** replaceafill has quit IRC22:11
*** replaceafill has joined #schooltool22:15
*** menesis has quit IRC22:22
*** ignas has joined #schooltool23:16
*** khildin_ has quit IRC23:19
*** ignas has quit IRC23:24

Generated by 2.15.1 by Marius Gedminas - find it at!