Hi,
A teacher of computer engineering who has taught various courses since last 6 years writes this.
Please re-phrase the choices. It is very hard to understand.
Thanks
An old fan of INRIA from CVML 2012 summer school.
2 Likes
Could you provide more information regarding which sentence is problematic?
Could you also provide a suggestion such that we can make the adjustment? Unfortunately, we are unfortunately not native speakers and are prone to make mistakes.
Thanks for the kind reply. Unfortunately, I am also not a native speaker due to which it is hard to understand “Choice:2” and “Choice:3”. It is the definition of the term “Substantially better” coupled with “at least 7” that is causing difficulties. For me it would much easier to understand something like.
→ For a given number of Neighbors, pre-processor ‘X’ performs the best or pre-processor ‘Y’ performs the worst
→ Similarly, for a given pre-processor, Neighbors ‘a’ are better than Neighbors ‘b’.
It may be my style of asking Multiple Choice Questions (MCQs) that might not be aligned with your course objectives, but I sincerely think that this question needs some updates.
1 Like
I see. The difficulty here is the definition of better. Since we don’t have a single score (because of the cross-validation), then we don’t have a simple comparison. Comparing the mean score without looking at the standard deviation of the scores would then be also wrong: there always be a model performing better for another in terms of the mean score while it could only be due to the randomness of the cross-validation.
A proper statistical analysis could be to do a statistical test but we don’t want to enter into this level of details in the MOOC because it requires a stronger background in statistics.
In the previous MOOC version, we were asked to look at the standard deviation of the models together with the mean and study the overlap of these standard deviations between models. However, it was reported to be confusing and not stable enough due to the randomness of the cross-validation.
So here, we decided to use a per-fold comparison and have a simple heuristic on what we can define as “substantially better” (we cannot use the term significantly better because we don’t do the proper statistical analysis).
Sorry for the long answer but it gives the full background to understand why we have a tricky formulation. From this, we are open to improving the formulation but using only “better” will not be enough here. I am still convinced that we can improve the formulation. Regarding the per-fold comparison, I would say that I am happy to report that we did not get negative feedback in comparison to our previous approach (based on score covering).
Hi, I also add a feedback for this question (had a hard time with it, finally validated with little luck):
I had to adjust a pd.DataFrame to explore the mean_score against the 2-dim parameters. While one of them is the type of preprocessor, I had to convert to string before filtering (and I also used df.rename_axis to get clearer aspect of presentation).
Answer c) seemed to work with 9 values above for 5N against 51N in a cross-validate (cv=10), but I think I’ve misunderstood here the inner and outer cv: already used the outer one at that point, while it seemed to be waited at q6 only. Maybe a hint about this aspect may help.
best regards…
I must agree, from N 4 on is very difficult to follow.