Austrian Research and Technology Report 2020

4. RTI Evaluation Culture and Practice
187
selection phase and a development phase. This pro-
cess resulted in five short applications for collabora-
tive R&D projects, three of which received positive 
assessments from the mentors during the workshop. 
Collaborative R&D projects were just one of the pos-
sible funding instruments – others included explor-
atory studies and R&D services. The short proposals 
were developed into full proposals by November 
2018, which in turn all received a recommendation 
for funding.
Results of the evaluation
The evaluation shows that the sandpit method clear-
ly results in more interdisciplinary sets of teams than 
those found in traditional R&D projects, and that 
stakeholders joined forces who “should have long 
since been working together”.
202
It was possible to 
successfully integrate individuals with non-technical 
expertise into the consortia. More diverse teams 
would have been possible if more suitable applica-
tion and implementation partners from companies 
and research institutions had applied for the pro-
gramme.
The sandpit method, involving a multi-stage selec-
tion procedure plus a creative process, enabled above 
all lateral thinking in the creativity phase. However, 
this creativity could not be translated into the further 
phases of the process, as there was not enough time 
and space from the participants’ point of view for de-
tailed development of ideas and for these to be 
transferred into short proposals. This is also why the 
creative process was unable to help the Austrian Re-
search Promotion Agency (FFG) in selecting better 
projects on issues that required a high degree of in-
terdisciplinary cooperation and/or new approaches in 
order to be solved. Nevertheless, the Ideas Lab has 
resulted in the selected teams pursuing more inter-
disciplinary projects, approaches and working meth-
ods than is the case in other Austrian Research Pro-
motion Agency (FFG) programmes. The selected proj-
ects also had a strong interdisciplinary composition. 
202 See Geyer and Good (2019, 10).
However, the mentors did not identify any radically 
new approaches or contributions that were of partic-
ular relevance in helping to solve problems.
The requirements for applicants outlined above 
have proven to be effective. This was not as true for 
the criteria for project selection because the need 
for application orientation made it difficult for partic-
ipants to pursue unconventional project ideas. It was 
not until the Austrian Research Promotion Agency 
(FFG) made it clear to them, that they realised that 
the criteria for project selection related to the possi-
ble funding instruments were a decisive specification 
in the openly worded call for proposals. In any case, 
there were indications that the intensive exchanges 
in the workshop resulted in new contacts between 
the highly diverse participants.
New stakeholders were also found for the Austri-
an Research Promotion Agency (FFG) from among 
the applications for participation in particular, al-
though these were less successful with the selection 
for the Ideas Lab and with the short applications.