List Comprehensions Are For Lists
Posted:
List comprehensions in Python are a great way of expressing a list but, as the name suggests, they are for lists and not purely for iterating over an iterable and calling some method.
If you find yourself writing:
[obj.some_method() for obj in my_objects]
then you actually intended to write:
for obj in my_objects:
obj.some_method()
I know, the first is a one-liner which feels exotic compared to an old-school for loop.
But the second version expresses what you want to do: call a method on each object and ignore the return value of that method. In the first version you're constructing a list, storing the return values and then forgetting about the list altogether because you're not assigning to a variable.
In CPython 3.3 both cases produce similar bytecode but a list is still constructed unneccessarily. Given that Python is designed for readability the second one expresses better what you're doing. If you do need to store the return values in a list then you can rewrite the code later.
Similarly, the same argument applies to the following construct:
map(lambda obj: obj.some_method(), my_objects)
If you need the constructed list of return values then you can rewrite it as a list comprehension. If you don't need the list then you can rewrite it as a for loop.
My First Coursera Course
Posted:
With Coursera and other online learning platforms gaining in popularity I decided to give it a try as well with the course Science, Technology, and Society in China I: Basic Concepts by Naubahar Sharif of the Hong Kong University of Science and Technology.
Why this course?
It is a very accessible course which defines and explains the concepts in a clear way. It builds well upon my background knowledge of the course Philosophy of Computer Science that I followed back in the day at an actual university.
Prior to the course I was familiar with the notions of falsification and paradigms (i.e., what constitutes science and the methods of science) and I had a general understanding of technology. But it turns out a lot more can be said about the relationship between technology and science within society and how that leads to innovations.
I picked this course because it would give me a better insight on technology and innovation as viewed within a society, in particular a non-Western society such as China. It also makes sense to study how innovations come about as there are many developing societies that want to improve their ability to innovate.
It is not uncommon to hear of new startups in Silicon Valley that strive to "disrupt their market" or to refer to product features as "innovations" when that is mostly a debatable opinion. In this light it is interesting to learn that scholars have actually studied what innovations are, how they occur and why doing the same thing in another country is also considered innovative.
On following an online course
Within hours of the start of the course there were already people asking in the forums for clarification about the first assignment. These questions could have easily been answered if you had actually viewed the lectures. At first I thought this was part of typical internet drama: students unwilling to do the work necessary to complete a course.
Based on the question's formulation that is still the most likely explanation. However, it occurred to me that some people may not have an adequate internet connection to view the lecture videos as in most Western countries. For me this course is mostly intellectual curiosity but for others it makes sense to take this course to improve their business relationships with China by having a better understanding of their society.
If it takes an evening to download each lecture then it makes sense to ask for the correct video in the course forums right away (i.e., the lecture video that explained the concepts mentioned in the assignment). Nonetheless I was happy to see the bar being raised for the second and third assignments to make sure decent efforts are put into completing the course.
Summary
I have just completed my peer evaluations which ticks the final box towards completing the course. It's difficult to assess your own submissions, especially when it's far more subjective than assignments given in most beta sciences, but I think I'll pass the course.
Perhaps the most striking example of innovation in this context is that online higher-education courses make university-level lectures freely available to anyone worldwide. Speaking in academic terms, the question of whether that is a "disruptive innovation" is left as an excercise for the reader.
Why Wikidata is Great for Wikipedia
Posted:
A recent Hacker News post made the Wikidata project more widely known in the tech community. It also created some confusion as the relevance of the project was not widely understood. I think Wikidata will easily be among the best projects for Wikipedia in 2013 and the future so it's good to explain why that is.
Wikidata solves a long-standing issue
First of all, it solves a long-standing issue that plagued many Wikipedia articles. Wikipedia's goal is the make knowledge widely available in many languages and many articles are indeed available in multiple languages.
Next to each article is a list of links to articles about the same subject in a different language. These links are manually added by specifying the language code and the title of the article. For example, if I want to include a sidebar link to the Wikipedia article in the English language I would include:
[[en:Wikipedia]]
where en indicates the language code of the project. However, these
so-called interwiki links need to be added to every language. So every article
stores interwiki links to all other articles about that subject.
This is fine as long as 1) nothing needs to change and 2) each article is unambiguously about the same subject. And this is where the problems begin: often an article would be renamed and then all interwiki links on all other Wikipedia projects would need to be updated.
This was not manually feasible so people developed bots to do this. Many bots would regularly update articles across all languages purely to keep interwiki links in sync.
With bots doing the hard work, all should be good, right? Not really because human error often introduced links to articles about a very closely but not exactly the same subject. For example, the article New York might describe the city on some Wikipedia projects while it discusses the state on others.
Bots don't understand this so they'll gladly review an article's interwiki links and rampantly copy across any missing links to other articles. This ensures interwiki links are the same in each language but it also means errors are propagated. Even worse, it requires human intervention to fix any incorrect links. If a Wikipedia contributor in some exotic language introduced a mistake in the interwiki links it would often propagate to other projects until it was manually fixed, everywhere.
Wikidata solves this by having one page per subject and by storing the interwiki links there. So instead of letting articles link to all other languages they now fetch the interwiki links from their corresponding Wikidata page. This means interwiki links can be updated in one place rather than in all places.
Long-term benefits of Wikidata
Now that we can have a unique page per subject we can build upon that foundation. The Wikidata page stores the interwiki links but it can also store facts about the subject. This makes the Wikidata project even more useful to Wikipedia.
For example, for cities it could store the number of inhabitants per census and for famous individuals it could store the date of birth, date of death and other relevant facts.
This makes it easy to:
- Generate the information box at the right side of many Wikipedia articles
- Produce tables with relevant information
- Make it possible to generate specific lists (e.g., all cities that had one million inhabitants in 1950).
That last point is worth elaborating upon: Wikipedia's articles are categorised so it's currently easy to find people born in a particular year but it's not so easy to find, e.g., all cities with a mayor born in 1950.
In a way the information is currently on Wikipedia but it's not so easy to find as it requires the reader to manually connect these facts together. The existence of Wikidata allows Wikipedia to turn into Semantic Wikipedia and that makes knowledge even more freely available than the project already does.
Not DoingSomething Yet
Posted:
Being new to London I thought it would be a good idea to try various approaches to meeting new people here. I'm not into online dating but I thought the concept of DoingSomething.co.uk would be a fruitful way of meeting new people. The site places emphasis on the activity rather than the person which makes for a different experience.
Great idea but the execution and implementation of the idea can certainly be improved. After signing up and clicking around on the website it quickly became apparent that, technically, there's still some work to do.
- It's not possible to change your gender, address and date of birth after signing up. If someone moves to a different address or, indeed, changes gender, then surely that should be possible right? And date of birth could be incorrectly filled in at signup.
- The "How to Use" page says that you can't send messages unless you're subscribed (i.e., becoming a paid member). Well, I don't have a subscription but I was able to send messages to other people.
- If you don't fill in any message and click "Send" it will send the placeholder text as message (i.e., "Write your message here").
- If you double-click on the "Favourite" button, the person is added twice to your list of favourite people.
- If you double-click on the "Send message" button and then close the message popup dialog the website still has the coloured background overlay.
- If you "Favourite" a person and then click "Remove favourite", the counter at the top still goes up so it shows you have two favourites, although you actually have none.
- I can't seem to make the unread count for "Sounds good" go down, even after archiving those messages from those users. Even after refreshing and archiving it remains the same.
- On the "100 Must Do Dates" page it shows the "Join Now" button even though you're logged in.
- If you use the "Reset Password" feature then your new password isn't checked for minimum length or other constraints.
What surprises me is that the company has been in business since 2011 and presumably has paying members yet a rather cursory glance reveals these technical issues. I'd say these issues are not hard to find so the end-user experience of others must be affected by it as well. It also doesn't build trust to actually become a paid member of the site.
On the other hand these issues may have been fixed by the time you read this and knowing the technical issues is always the first step towards fixing them.
How to Mock Logging in Python
Posted:
As you become familiar with software testing and unit testing it can be necessary to assure yourself that information will be logged when your software platform raises an exception or behaves unexpectedly.
In other words, you've developed a new feature and you want to make sure that certain calls to Python's logging module are performed in certain cirumstances.
That's where the mock module comes in:
it allows us to mock the Python logging module and assert that certain calls
are made. If you're using Python 3.3 (or higher) than you can import from the
unittest.mock (documentation)
module rather than installing mock from PyPI.
Example of mocking in unit tests
Let's say our code looks like this:
import logging
def check_value(data_dict, value):
try:
return data_dict[value] > 10
except KeyError:
logging.warn("Data does not contain '%s'", value)
return False
What we wish to test is that logging.warn is called when the KeyError
is raised. So in our unit test we call check_value with parameters in such
a way that the KeyError is indeed raised. For example:
import unittest
from my_module import check_value
class MyUnitTest(unittest.TestCase):
def test_check_value_logs_warning(self):
check_value({}, 'key')
The idea behind mocking is as follows: instead of calling the real
warn function of the logging module we call a fake mocked version and
that allows us to keep track of how many times the function is
called (if at all).
We can do that using the patch function of the mock library: this
specifies what we wish to mock and what the name of the mocked object should be.
We can now update our test by writing:
import unittest
from mock import patch
from my_module import check_value
class MyUnitTest(unittest.TestCase):
@patch('my_module.logging')
def test_check_value_logs_warning(self, mock_logging):
check_value({}, 'key')
self.assertTrue(mock_logging.warn.called)
if __name__ == '__main__':
unittest.main()
This unit test passes as the assertion is indeed true. If you modify the code in a way that it no longer logs a warning then you'll find that the assertion in the unit test fails.
Participating in the International Four Days Marches Nijmegen
Posted:
After not making the cut last year I was quite pleased to receive an email today that I'm among the debutants allowed to participate in the 97th edition of the International Four Days Marches Nijmegen - more colloquially known as the "Vierdaagse".
Given my age and my gender I'll be walking the 50km routes each day which means it sums up nicely to 200km in four days.
I'll have to prepare properly as the longest walk I've done so far on one day is the Tongariro Alpine Crossing in New Zealand which is a little less than 20km. To be fair, that's a different challenge altogether as it is across vulcanic terrain with steep climbs, descents and changing weather conditions.
In contrast the Vierdaagse is an event in and around the city of Nijmegen across the typically Dutch flat landscape with a backdrop of little towns and distant green panoramas. The distance and the possibly hot or rainy weather mean it is still not wise to participate without training (although some do try every year).
The event is as popular as ever with over fifty thousand people signing up this year and forty-six thousand allowed to participate. This easily makes it among the biggest events in the Netherlands and also one of the events that you always hear about when growing up.
With about three months to go I'm going to look into some walking routes around London. The distance from east to west in Greater London is about 50km so that gives you an idea of the task ahead.
Photos of London in Spring 2013
Posted:
Today was one of the first sunny days of spring in London and after several months of cold weather a change was much welcomed. I decided to carry my digital camera with me to take some sunny photos of London's diverse architecture. It doesn't take long to realise that hundreds of years of history have given London a unique mixture of historical and modern buildings.
A stroll through various neighbourhoods and towns in London brings you to busy streets, quiet mews and narrow passageways that are sometimes no more than a pedestrian wide.
Below is a selection of the photos taken today:
Red Place, London by Simeon Visser
Phonebooth in London by Simeon Visser
Life and Art in London by Simeon Visser
On Photography and Travel
Posted:
About a month ago I uploaded a selection of my better travel photos to 500px.com (see my profile there). It's far from my entire photo collection but I thought it would be nice to put some online rather than letting them collect digital dust on my hard drives.
It's also quite clear that a basic digital camera, some sunshine, decent composition and nice scenery are no longer sufficient to produce a work of art. My uploaded photos are indeed what they are: carefully selected holiday photos but not masterfully edited works of art made with the finest equipment. I don't have a trained eye for this but I do feel the ratio of edited versus unedited photos is skewed towards the former.
My previous camera was the Samsung L370 and my current camera is the somewhat better Coolpix S8200. I have to say I barely know more than the automatic feature of the camera but I'm planning to remedy that in the future.
Fresh New Look
Posted:
Although it's late March and London hasn't quite shown what spring should look like I thought it would be good to do some spring "cleaning" of my website. That means saying goodbye to my homebuilt combination of PHP and Markdown and saying hello to Nikola, a static site generator written in Python.
As Nikola supports Markdown it was fairly easy to convert my old website to this new setup. I can now focus on writing the content and generate the whole website with a single command. Some new features are the posts archive and the ability to tag posts.
The new theme is still quite Bootstrap-y but at least it's one of the themes (Readable) from Bootswatch rather than the default Bootstrap theme.
I have configured the .htaccess file of the website so old links should
still work. If you have come across a link that doesn't work anymore please let me
know on Twitter.
Using 'else' in Python
Posted:
Python allows you to use the 'else' keyword in places where most other
programming languages don't. I assume that you're familiar with the
basic if/else where the block after the 'else' is executed when the
condition in the if-statement isn't true.
In Python you can also add 'else' to a for loop, a while loop as well
as a try/except section. I was aware of the ability to use 'else' in combination with a for
loop but I only learned of the try/except/else combination a few days ago
so perhaps it's worth spreading the word about this.
Using 'else' in combination with 'if'
For completeness it's important to discuss the basics of the if/else first
as the other uses of 'else' in Python differ slightly differently from this.
if condition:
# executed when condition == True
...
else:
# executed when condition == False
...
Despite the familiarity of this construct, it's worth noting that the reason
why the else block is executed is explicitly mentioned in the
code (i.e., the condition in the if-statement).
For the other uses of 'else' you'll see that this isn't the case. You'll need to have read a Python guide (or this article) to know when those 'else' blocks are executed.
Using 'else' in combination with 'for'
This is where things become interesting as the else statement can lead
to cleaner code. It frequently happens that we wish to iterate over a collection
of objects and when a certain condition is met we break from the loop.
For example:
for car in cars:
if needs_repair(car):
send_for_repair(car)
break
In this example we iterate over a collection of cars and we stop when we've found a car that needs a repair. No further cars will be examined when the first broken car has been found.
Now, what if we wish to take a certain action
when we haven't found any car for repair? We could introduce a
variable car_found_for_repair for this purpose:
car_found_for_repair = False
for car in cars:
if needs_repair(car):
send_for_repair(car)
car_found_for_repair = True
break
if not car_found_for_repair:
# do something
...
Python allows an elegant solution by adding an else
statement to the for loop. The block of code after the else is executed when
we have not performed a break in the loop. The code now looks as follows
and it behaves the same:
for car in cars:
if needs_repair(car):
send_for_repair(car)
break
else:
# do something
...
In other words, when the for loop completes successfully (i.e., without
being exited by a break statement) the else section is executed. But
when we break from the loop at some point then that section won't be executed.
Using 'else' in combination with 'while'
This is feature in the Python language that I only discovered when reading the grammar specification of the language. This construct may not be that popular or, at least, I have not seen it in the Python code that I have read over the years.
Similar to the for loop you can add an else statement to a while loop
and it'll be executed when a break has not been performed in the loop.
I'll leave the example out as it would be similar to the above one.
Using 'else' in combination with 'try' and 'except'
The basics of a try/except section in Python consist of:
try:
# run some code
except ValueError:
# catch exception when it is raised
This means the code within the try is executed and when a
ValueError is raised execution continues in the except block. Some of you
will know that a finally block can be added to execute code after a
try/except regardless of whether an exception was raised:
try:
# run some code
except ValueError:
# run this when ValueError is raised
finally:
# run this after the code and any exception handling code
Similar to the for loop example above, what if we wish to execute some code
when no exception has been raised? We could introduce another boolean variable
for that purpose but Python allows you to add an else block:
try:
# run some code
except ValueError:
# run this when ValueError is raised
else:
# run this only when no exception is raised
finally:
# run this after the code and any exception handling code
The else section is only executed when no exception at all is raised.
In this example we are only catching ValueError exceptions. When, for example,
an IndexError is raised then Python will run the try block up until the
exception, the finally block and the ValueError won't be caught (yet).
Contents © 2013 Simeon Visser