ACM SIGCHI-Sponsored Summer School On Crowdsourcing At XJTLU


Summer School On Crowdsourcing At XJTLU

Kostantinos Papangelis


The proliferation of the Internet has raised a new type of work-related platforms, known as Crowdsourcing platforms. One might have already heard of AirBnb, sourcing rooms, and Uber, sourcing car rides, from the crowd. Nevertheless, these two platforms are only the tip of the iceberg. Amazon Mechanical Turk [] and CrowdFlower [] source image annotations, content moderation, sentiment analysis and video transcription, and source the crowd’s creative ideas for product sketching and graphic design, and sources participants for academic research, to name just a few. Crowdsourcing platforms impact several salient aspects of life, ranging from social to technical and everything in-between; especially in developing countries where they form the basic form of income generation for thousands of people. We tried to address those aspects in a week-long, ACM SIGCHI sponsored- summer school that took place at Xi’an Jiaotong-Liverpool University in Suzhou, China from the 10th to the 14th of July, 2017.
The school included lectures, assignments with supervision and feedback from scholars, a visit to Netherlands and Shanghai-based company Design2Gather that utilises crowdsourcing in its design processes, and a hackathon that concluded the school. Lecture topics included: guidelines for designing crowdsourcing platforms; crowd motivators; ethical implications; collaborative aspects in crowdsourcing; location-based crowdsourcing; and the Chinese context of crowdsourcing.
Below we provide a brief overview of each lecture and of the main activities.

Figure 1: Dr. Khan from Eindhoven University of Technology (TU/e) kicked off the summer school presenting on how to evaluate the design of crowdsourcing platforms.

The design of such platforms appears as a key component for their success in enabling more people to become part of this new way of working and to support different kinds of work. Related research has explored how to design such platforms but currently, designers of such platforms lack a systematic way to evaluate their platforms against good practices and compare to their competitors. Dr. Khan organized a workshop during the school on how to evaluate crowdsourcing platforms based on work developed at the Industrial Design Dept. in Eindhoven University of Technology.

Figure 2: Prof. Kostakos from University of Melbourne presents how to harness smartphone data for crowdsourcing research.

Professor Vassilis Kostakos (pictured above) from the University of Melbourne, Australia, delivered a lecture on using smartphones for crowdsourcing research projects:
“Most research today in Human-computer interaction (HCI) takes the following approach: we say, ‘We study people, we understand people, and so we can build better technology’ – cars, smartphones, etc.,” he said. “I’m trying to convince everyone that we can and should do the exact opposite – if we study technology we can understand people better.” Smartphones allow researchers to do ‘non-invasive’ sensing – I.e. collecting data directly from large numbers of individuals ‘in the wild’ rather than from a small number of test subjects who have been connected to expensive and uncomfortable sensing equipment in a laboratory.

Figure 3: Dr. Lykourentzou, senior scientist at Luxembourg Institute of Science & Technology presents how to tackle coordination and collaboration issues in crowdsourcing.

Ioanna Lykourentzou introduced students to two frontiers of crowdsourcing research: collaboration and coordination. On the first part of her lecture (collaboration) she explained how current commercial online labor platforms use only a part of the crowd’s potential, mostly limiting participants to simple, repetitive tasks like annotation. She explained how collaboration at-scale can enable people to utilise their higher-order skills, like creativity, and presented a novel approach to build highly performant crowd teams, based on personality compatibility. On the second part of her lecture (coordination) she discussed scalability and performance issues, explaining that large-scale collaboration is only feasible by putting in place task assignment optimisation algorithms that can recommend to crowd participants which jobs to undertake. She finished her lecture with a hands-on exercise where the students experimented with the team building methods presented in the first part of the lecture.

Figure 4: Bram Broeken, CEO of Design2Gather, a crowdsourcing company based in Shanghai, China talks about the advantages and challenges of crowdsourcing for product design purposes. On the issue of how can companies protect their IP in crowdsourcing platforms Bram’s answer was clear: “one has to always look forward; in an era when innovation moves at the speed of light, one has to always think of the next step in improving their products and this is where crowdsourcing platforms can help!”

Figure 5: In addition to students from China, students from Australia, India, Turkey, USA and the Netherlands participated in the school.

The event was organised by Dr. Konstantinos Papangelis, lecturer in the Department of Computer Science and Software Engineering at XJTLU, and Dr. Khan, assistant professor in the Department of Industrial Design at Eindhoven University of Technology, the Netherlands. “Crowdsourcing is an emerging trend that promises to transform collaboration and creation in several domains,” said Dr. Papangelis. “These include creative work, research, business, and cultural cooperation. As such it represents a field of immense interest for research but also a tool that students can leverage during their studies.”

Figure 6: Dr. Papangelis, organizer of summer school presenting the schedule.

Figure 7: The school included daily social activities.


Managed by Comms