Networking across organizations is considered a very important component when transferring and sharing new knowledge and technologies with stakeholders. Networks are known to create trust amongst their members, facilitate the navigation of bureaucratic and political hurdles and generate ownership of the shared knowledge. Assessing the effects of knowledge networks in the successful transfer of technologies vis-a-vis top-down traditional technology transfer has become increasingly important in order to justify the rather resource intensive network activities (Barr et al., 2009).
However, it is extremely difficult to attribute and quantify the impact of new networks on the achievement of their development aims. Usually, impact is determined by using secondary information about the operation of networks. Networks can be evaluated on the number of members (density), the kind and frequency of interactions (vibrancy), the quantity or quality of exchanged information, their measurable outputs, their visibility or their popularity. Some of these parameters are quantifiable while others are based on anecdotal information or triangulation.
Network density and vibrancy
Basic network analysis looks at a few key parameters that can be collected through surveys:
- Number of links amongst organizations or individuals within organizations (nodes).
- Type of connections within a network such as links amongst similar organizations (e.g. research-research) or between different types of organizations (e.g. research-development, government-NGO, etc.)
- The quality of the connections which can be assessed by looking at, for instance, the quality of interaction (i.e. what kind of interaction is there: personal or impersonal, active or passive, one-to-one or within groups, meetings and workshops), and
- Frequency of interaction (daily, weekly, monthly, etc.).
The result of such a basic network analysis can be presented through visualization, using tools such as NetDraw (Borgatti, 2002), to provide an overview of the network (Figure 1). The information value of such a network visualization is often weak, as it is difficult to add qualitative parameters into the graphic without overloading it. This is specifically a problem of networks with a large number of nodes. The picture might become very crowded because of multiple interactions. It is also possible that one participant, a super networker, has so many contacts that others become almost invisible. This phenomenon is not uncommon in social networks.
One of the key difficulties in evaluating knowledge networks is that detailed data collection is extremely resource intensive and thus the database from which networking information is to be analysed is often incomplete. However, although many times the information is very superficial, even this limited information can be an important source of knowledge about the paths of communication and knowledge transfer from which to infer the network's vibrancy.
Quantity and quality of outputs
o evaluate the qualitative attributes of a knowledge network, it is possible to assess its actual information outputs such as types and number of technologies transferred and to how many users, or number and kinds of information tools (such as manuals, newsletters, guidebooks, videos, etc.) shared with the network members or beyond. In a knowledge network, this may be done through evaluation of training events, specific Knowledge, Attitude and Practice (KAP) surveys of trainees, and evaluating KAP again after a certain amount of time has passed after the training. These surveys are powerful tools to assess the amount of new knowledge that is being used and transferred to others. As for other surveys, however, the database returned through them is often small and incomplete.
Visibility and popularity
Visibility and popularity of a knowledge network are also important measures of its quality and usefulness to its members. These parameters can be assessed by evaluating the interactivity and acceptance of websites and other information media, and by soliciting open feedback. The utility and popularity of a website can be measured by the number of website visitors and page visits as well as their geographic spread. New social media have also popularized the use of 'Likes' as measures of success. More challenging is the analysis of reflection sessions and open-ended comments, which are useful to provide a general overview of a network's social fabric.
Case study: SATNET Asia
An evaluation of the Network for Knowledge Transfer on Sustainable Agricultural Technologies and Improved Market Linkages in South and South-East Asia (SATNET Asia) took place at the end of its 3-year implementation phase. The SATNET Asia project was funded over the period 2012-2015 by the European Commission as one of the projects within its regional programme 'Technology Transfer for Food Security in Asia' (TTFSA). The aim of SATNET Asia was to improve regional trade and agricultural productivity by making available information and knowledge about trade facilitation measures and sustainable agriculture technologies through fostering networking across the regions of South and South-East Asia. A baseline study carried out at the beginning of the project (SATNET, 2012) provided an important background for the final evaluation.
The final evaluation of the SATNET Asia knowledge network provided data on the network's density (numbers of ties), strength (multiple ties) and regionality (intraregional versus national connections) as well as information about links across types of organizations and frequency and type of interaction amongst them. In addition, information about training events organized through the network, including via the KAP surveys conducted immediately after the event and 6-12 months later, was evaluated. To assess the visibility and popularity of the network, information about utilization of social media tools was also collated. The evaluation of the quantity and quality of the trade facilitation measures and best practices information collected and shared by the network are beyond the scope of this article.
Key parameters of network density and vibrancy are shown in Tables 1 and 2. They indicate moderate network activity, still mainly amongst national partners with rather limited intraregional links, and limited interaction across organizational types.
The evaluation of the training events indicates a high quality of the network's core knowledge sharing activities and considerable potential for sustainability (Table 3).
A look at the social media statistics indicates a moderately lively network (Table 4).
Whilst networks and their quality in terms of density and vibrancy can be analysed and visualized, their effect in transferring knowledge successfully remains difficult to quantify and relies on anecdotal evidence.
Anecdotal evidence from the SATNET Asia stakeholders confirms the notion that being a member of a knowledge network enhances the chance of participating in information exchange and that individual learning experiences contribute to innovation at local scales. Training and capacity-building of intermediaries are valuable elements of transferring knowledge. The SATNET Asia network has carried out a large number of training events over the past 3 years and has trained well over 1,400 people. A large percentage of these trainees are, in turn, sharing their newly acquired knowledge with others. Although it is not possible to quantify the amount of learning that is being transferred, these connections can be visualized and thus provide a first step towards understanding the pathways of innovation.
Learning and communicating new knowledge is only one part of technology transfer; for the new technologies to be applied there needs to be an open, facilitating environment. Here too, networks can help to build trust across different levels of stakeholders, especially across research, outreach and policy agencies which too often do not engage sufficiently. This is a long-term, ongoing and very personal process. It is important that these processes continue beyond the immediate steps of creating a new network.
(References will be made available upon request)