Trade Sustainability Impact Assessments: Revisiting the Consultation Process

Trade sustainability impact assessments are the main tool used by the European Commission to inform policymakers on their potential effects and engage with stakeholders. These have two elements: a model-based quantitative analysis of likely economic impacts of a trade agreement and an associated consultation process. Both elements include a focus on a broad range of non-trade issues ranging from sustainable development to human rights. This paper argues that current approaches can be adapted to provide better guidance to policymakers on what stakeholders regard as priority nontrade issues and making the consultation process a more effective input into the design of trade cooperation. Inclusion of closed end questions on a broad range of non-trade policy objectives with scaled responses, complemented by deliberative polling of a representative sample of survey respondents, can identify a small set of priority non-trade issues and views on how to leverage a trade agreement to mobilize attention and implement policy measures that can address them most effectively.


Introduction
The Treaty of Lisbon calls for European Union (EU) trade and investment policy to support and promote the Union's values and standards relating to human rights, labor rights, environmental, health and consumer protection, sustainable development, as well as broader objectives of protecting people and the planet (Van den Putte and Orbie, 2015).Realizing these non-trade policy objectives (NTPOs) requires coherence across policy areas as well as commonality of action (EC, 2015a; Apiko et al. 2019).The 2020 review of EU trade policy (EC, 2021) reaffirms the need to balance pursuit of material interests through trade agreements with EU values.The challenge in operationalizing these principles is significant, given that it will be context specific.It is very unlikely that a 'one-size fits all' approach to the design of trade agreements and external cooperation will be appropriate.
There are two basic aspects of EC efforts to implement an external trade policy that satisfies the prescriptions contained in EU law.The first is ex ante: determining the design and substance of policy interventions to maximize the prospects of achieving EU non-trade goals efficiently.The second is ex post: associated with implementation of agreements, complementary actions (e.g., technical assistance; development aid), and monitoring and evaluation -assessment of the extent to which NTPOs are achieved.In this paper we focus on the ex-ante dimension, while recognizing that ex ante analysis can (and should) be relevant to ex post evaluation by identifying key performance indicators and baselines against which to assess the results of implementing trade and aid policy instruments. 2e main ex ante tool used by the EU is the trade sustainability impact assessment (SIA) to determine the potential impacts of proposed trade agreements and informing negotiators and policymakers on the design of trade agreements and complementary (flanking) measures.A trade SIA has two elements: a quantitative analysis of the potential economic effects of a trade agreement on European and partner country industries and consumer welfare, and a complementary consultation process (CP).The thematic coverage of SIAs and the CP includes European values as reflected in Article 21 of the Treaty on European Union, the Charter of Fundamental Rights of the EU, and EU guidelines to promote human rights through its external actions (Council of the European Union, 2012), as well as commitments made by EU member states enshrined in international law -e.g., the Universal Declaration of Human Rights and the International Covenant on Civil and Political rights. 3The CP includes a focus on stakeholder perceptions with respect to the broad range of non-trade issues and objectives that are more difficult to include in ex ante modelling of a trade agreement.The CP is intended to provide transparent two-way communications between the European Commission (EC) and stakeholders, informing the trade negotiating team of issues and concerns that may influence the eventual approval of the outcome of negotiations by the Council and the European Parliament.
In this paper we discuss the SIA process, with an emphasis on the role of the CP, contributing to discussions on how to strengthen the implementation of Trade and Sustainable Development chapters in EU trade agreements, and by extension, similar provisions on NTPOs in trade agreements negotiated by other countries.Although the transparency and inclusiveness of the SIA/CP process has increased over time, consistent with critiques of the 'blueprint' approach used by the European Commission based on the template sustainable development chapter of EU trade agreements (Harrison et al. 2019), greater effort is needed to move away from a one-size fits all approach to NTPOs in trade agreements.We argue that SIAs do too little to identify issues that are more (most) important to stakeholders in the EU and partner countries.In practice, SIAs include a long list of non-trade issues.Many of these end up being superficially treated, reducing the usefulness of the assessment.Any trade agreement will have heterogenous effects for different socio-economic groups in the EU and partner countries, and the relative importance of non-trade issues will vary.Hence, it is critical to quantify these potential impacts and/or contextualize the magnitude of the effects for different economic activities and agents.
We argue that the CP should include a tailored survey of the population of stakeholder groups that serves as an input into and provide the basis for the application of deliberative polling techniques to identify a small set of priority issues of concern in each trade negotiation context. 4A corollary of our approach is that it accepts that priorities may vary across partner countries. 5Recognizing that the importance of prioritizing certain sustainability/human rights concerns for specific trading partners is often discussed and criticized,6 highlighting a limited number of potential effects that are of significant interest to many stakeholders would increase the salience of SIAs by focusing attention on specific NTPOs where a trade agreement could potentially have a positive impact.Greater specificity would not only help inform the content and design of agreements but help identify where complementary instruments and measures are needed.Doing so arguably could increase the coherence and effectiveness of EU external cooperation, as well as increase accountability for results by generating agreed baselines against which to assess performance once agreements are concluded and have been implemented.
The remainder of the paper is organized as follows.In Section 1 we briefly describe the role of the SIA in EU trade policy formation.Section 2 discusses the consultation process that is part of the SIA.Section 3 focuses on the methodology of the CP, using human rights as an example.Section 4 discusses how the use of surveys and deliberative polling as elements of the CP can generate more specific guidance to negotiators and decisionmakers while ensuring representativeness and building a bridge between the ex-ante CP and the 'downstream' implementation dimension of pursuing NTPOs in trade agreements.

The SIA in the EU trade negotiation process
The full process of negotiating and implementing EU trade agreements involves more than 30 different stages and all three EU governance bodies: the EC, the Council, and the European Parliament (EC, 2012).
The stages (33 in total) can be divided into five main components.The first is preparatory, with the EC (DG-Trade) initiating the trade negotiation process, preparing a scoping report, and commissioning the trade SIA.The second is the mandate, where the EC seeks the authorization from the Council to start negotiations and if approved, the Council provides the EC with the negotiating directives.The third comprises the pursuit of negotiations between the EC and partner countries, with EC regularly informing the Council, providing intermediate reports, some of which are made public.The fourth step involves finalization and signature of a draft agreement, including legal scrubbing and translation of the texts.
The fifth and final component is ratification by the Council and the European Parliament, after which the agreement can enter full or provisional application until entry into force.
Although the full negotiation process is described as a chronological sequence, in practice the trade SIA is never concluded before the negotiations begin.In this regard, the SIA is not really part of the "preparation" process, but it is an ongoing exercise that is held after the mandate is given to the EC and it overlaps and may extend beyond the negotiations.Looking across different EU trade negotiations, in most cases the tendering process for the SIA only starts after the negotiations have been launched, and the SIA final report tends to be published two to four years after the negotiations commence.There is no recognition in EC descriptive documents that in practice the SIA and the negotiations overlap.This is a fundamental timing inconsistency as in principle the trade SIA should be fully contained in the first stage of the negotiation process, as the impact assessment is meant to provide guidance for the negotiations.Moreover, in its conceptualization of the sequencing of the negotiation stages, the trade SIA and associated CP are not mentioned as figuring beyond the preparatory stage, suggesting the SIA/CP process is not intended to provide feedback to the negotiations.This reduces the practical impact of the SIA on the negotiations as well as its potential role in building public support for the trade initiative. 7ere are detailed handbooks to guide the SIA process (EC, 2006; 2016).The most recent (2016)   handbook defines a range of activities and communication methods to be implemented by the consultants.These encompass the creation of a SIA dedicated website with accessible summaries of the stages of the SIA process, provision of periodic electronic newsletters, links to reports, and feedback mechanisms to permit users to provide inputs and suggestions.The latter include developing a survey questionnaire to pro-actively solicit information from the general public, SMEs and consumers.These are complemented with interviews, meetings, and roundtables with stakeholders.The SIA process has become more transparent over time, reflecting implementation of the various provisions to this effect in the latest handbook (EC, 2016).As discussed by Marx and van der Loo (2020), there is an inherent tension between transparency and the secrecy that is needed when it comes to the details of an ongoing trade negotiation process, but transparency and effective communications are important to garner public support for an agreement. 8st trade SIAs do not explicitly assess the country-specific results for each of the now 27 EU member states.The main analytical tool used is a Computable General Equilibrium (CGE) model, which can be set to provide this level of detail at the EU member-state level, but so far, only two SIAs (for CETA and TTIP) have presented results for each member state.The main reason is that for many trade agreements the impacts on the EU in the aggregate are negligible and thus there is no expectation that any EU member will experience non-marginal impacts.However, in some cases the impacts can be more substantial (even if still relatively small) and here it can be interesting to corroborate that this also holds for each EU member-state.In general, presenting detailed country-specific results can increase the transparency of the SIA process.

SIA consultation processes
There will be winners and losers from any trade agreement.Assessing the distributional consequences of a trade agreement is a core task of the SIA model-based analysis of the potential effects of an agreement.Identifying groups that may confront significant adjustment costs should inform the substantive design of an agreement and the need for flanking measures.This is also an important function of the consultation process, which provides a platform for stakeholders to express concerns regarding possible negative effects of a trade agreement.Open communication and broad consultation processes are a critical input for the legitimacy of trade policy initiatives (Torriti and Lofstedt, 2012;   Kirkpatrick and George.2006).By being able to assert that a broad set of opinions has been considered that have informed policy decisions, an open consultative process is both an important (potential) input into SIA analysis and a complement to such analysis. 98 Although transparency is a desirable from a legitimacy and accountability perspective, it is not evident that it improves the quality of outcomes.Dudley and Wegrich (2016) analyze impact assessments used in regulatory procedures in the US and the EU and conclude that there is an absence of robust empirical evidence on the impact of transparency on regulatory quality. 9Buena and Thompson (2015) discuss how consultations during the formative stage of the policy process affect the bargaining success of the European Commission during the decision-making stage after new policies are put The first edition of the EC's SIA handbook (EC, 2006) divided the CP into an internal and an external part.
The internal process involves exclusively EC institutions, which provide comments and suggestions to the consultants that are contracted to carry out an SIA study, guided by a steering committee -consisting exclusively of EC representatives.The external element spanned interactions with a large and diverse set of stakeholders, representing the private sector, different levels of government; legislatures and political parties; and civic society: the media, religious organizations, academia, advocacy groups, trade unions, and civil society organizations (CSOs).This process was to be implemented both within the EU and in counterpart countries if a negotiation was with a developing nation.The 2006 handbook recognized that the CP is a complex undertaking that requires 'careful steering' to reflect and balance the interests of different stakeholders.The handbook specified that achieving this balance called for the contracted SIA consultants to conduct a "thorough stakeholder analysis" to identify all parties concerned with a trade agreement and ensure they are consulted in an evenhanded manner, engaging with both experts and the general public.EC (2006) gave consultants substantial leeway to design the CP but did not provide guidance on how to ensure the representativeness of the organizations that (are invited to) participate.A central concern pertains to the need to identify all (direct and indirect) stakeholders.The second edition of the SIA handbook (EC, 2016), issued ten years later, noted that "an open, transparent and wide-ranging consultation process is at the core of SIAs' analysis..." and that consultations "…are critical to ensuring the transparency, quality, credibility and legitimacy of SIAs by providing a dynamic and robust framework for interaction and dialogue with all relevant stakeholders."Although the selection and representativeness of the groups to involve in the CP remained unaddressed in EC (2016), it stresses that the goal is to "ensure an adequate and balanced coverage of all relevant interested parties during the consultation in order to be representative and avoid capture of the process by specific constituencies."Building on recommendations in an assessment of trade SIAs by the European Economic and Social Committee (EESC, 2011), EC (2016) requires consultants to identify key stakeholders in close cooperation with the EC, the EESC, the European Consumer Consultative Group and the Sectoral Social Dialogue Committees and to cooperate with national human rights institutions to identify vulnerable groups.
forward to legislative actors.More broadly on the importance of design of consultation processes to inform policies to pursue the SDGs, see Sénit, Biermann and Kalfagianni (2017).Homeyer et al. (2009) define five goals for public participation in policy making: credibility, legitimacy, trust, ownership, and improved policy performance.Each of these goals requires high levels of public knowledge, well established formal and informal participation mechanisms, and proper linkages for these mechanisms to inform policy.The authors stress the intrinsic complexities of distilling trade policy impacts on economic and non-economic variables of interest (social, labor, environment) and that this complicates achieving public participation goals.Use of umbrella organizations and/or multinational networks to identify and contact stakeholders and government officials in partner countries and creation of a `stakeholder database' using networks of government organizations, CSOs and civil society organizations can greatly facilitate the consultation process.Homeyer et al. point to an absence of formal systems to evaluate the quality or sufficiency of participation in CP, unclear participation objectives, weak links to the negotiation process and the technical (knowledge and capacity gaps) and logistical obstacles to effective public participation. 10They conclude that the CP should be based on a set of realistic and regionally tailored objectives and formal mechanisms to integrate the process into the negotiations.Three issues arise: (i) ensuring that the CP involves and engages with a representative sample of stakeholders; (ii) weighing the views and comments expressed during the CP by participants; and (ii) analyzing how changes in trade and the provisions of a trade agreement may (can) impact on different groups in society and specific NTPOs that were brought forward in the consultations. 11A CP that takes the directives in the handbook (EC, 2016) seriously and makes it a genuinely inclusive process that reflects the opinion and concerns of a very broad and heterogenous group of stakeholders is not only a complex resource and time-intensive exercise, but can easily result in a kitchen sink of issues and views.The EC's consultants, given a constrained budget and strict timeline, confront a major challenge if they want to involve all stakeholders and civil society organizations to generate a properly represented (and balanced) overview of concerns expressed and suggestions made.
Our reading of the literature, interviews with practitioners and experience in implementing the CP methodology is that it lacks sufficient guidance and mechanisms to complement inclusiveness and 10 The International Federation for Human Rights (FIDH) is similarly critical of several elements of the CP, noting that usually only one workshop is conducted at a local level, with 20-50 participants, insufficient to obtain the necessary input and ensure representativeness, that the selection process in respect of who can participate is opaque, and that the most vulnerable and indigenous people are not consulted (FIDH, 2015). 11Whether the strategy of linking trade agreements to NTPOs is effective is an important question we do not address here -we take as given that this a feature of EU trade policy.See Pelkmans (2021) and Borchert et al.  (2021) for discussion of the strategy; Ferrari et al. (2021) for a quantitative assessment.
representativeness with a methodology that identifies what a majority of stakeholder groups consider to be priority areas for attention and action.As discussed further below, the CP arguably does not do a very good job on these three fronts when it comes to NTPOs.It is not clear that in practice participants in consultations are representative.Even if they are, there is no explicit and transparent weighting mechanism that rank orders views and identifies a set of priority areas for action.Positions on issues taken in the CP may reflect perceptions of the direct economic impacts from an agreement on specific groups and economic activities that may not have a robust empirical basis.Views will be fraught with uncertainty as indirect second round effects may be unclear if they are considered at all.
Ultimately the balancing of different interests and effects of a trade agreement is the task of the Council and the European Parliament, both of which are appointed through democratic processes in and across EU member states.But for these institutions to be able to engage in informed deliberations and decision-making -including not only on the substance of a trade agreement but accompanying flanking measures -they should be able to rely on the SIA and CP to assist them in doing so.As it stands the CP arguably does not deliver what is needed, as it does not generate a "bottom line" assessment of stakeholder views that involves aggregation and weighting to identify the non-trade issues deemed most important, which are regarded as most susceptible to being addressed in a trade context, and which call for complementary measures that should accompany implementation of an agreement.

Practitioner perspectives
A practitioner survey by Fiorini et al. (2019) provides additional information on perceptions about the SIA consultation process.One of the questions asked states: "The design of trade agreements is efficiently informed by consultation of stakeholders."They find that 49% of respondents agree or strongly agree with this statement, while 25% disagree or strongly disagree.Most respondents therefore expressed a favorable view on the consultation process, notably academics and employees of EU institutions.However, two groups that are key to the CP were more negative.Of the five trade union representatives that responded, most disagreed or strongly disagreed with the statement and none agreed with it.Half of the 20 CSO respondents disagreed with the statement.International organization staff (19 respondents) also had more negative than positive views (47.5% vs. 37.5%).Respondents affiliated with EU institutions (39) had the most favorable views, while EU business associations (21) and firms (small, medium and large) have a majority of positive views regarding the consultation process.
Interviews with SIA consultants are consistent with the survey results.We interviewed a cross-section of the consultants that took part in past SIAs, jointly having participated in three-fifths of all EU trade SIAs done through 2019.Most interviewees considered that the CP influenced the design of SIAs and directly or indirectly influenced the coverage and methodology of the SIAs over time.Most did not believe the CP influenced the negotiating process, with some notable exceptions -e.g., the CP for the CETA negotiations made clear that investor-state dispute settlement was unacceptable for several stakeholders (Development Solutions, 2011).It was also suggested that the selection of CP participants should be formalized and made more representative, including CSOs, employer organizations, unions and labor representatives, as well as government officials.Suggestions included creating a standing body that includes representatives drawn from these groups, similar in nature to the Social and Economic Council of the Netherlands, to provide guidance to negotiators and decision-makers involved in a trade negotiation.Many practitioners called for greater direction on the quantifiable indicators that should be employed in SIAs.Echoing both evaluations and the policy research literature, a key recommendation was that the consultation process should be done before the beginning of the economic analysis and ideally, also before negotiations start.

Methodological weaknesses and challenges
EC (2016) calls for SIAs to include analysis of human rights, social, and environmental issues in addition to economic (sectoral) effects.By including these topics in the SIA methodology, it is implicitly assumed that trade agreements can affect in a meaningful way the situation in a particular country.This is a (very) strong assumption for many of the topics included.The treatment of NTPOs and values in trade SIAs and the CP provides limited practical use for policy recommendations. 12This is not to say that SIAs do not seriously engage with the subject.The 2019 final SIA report in support of the negotiations for the modernization of the trade part of the Association Agreement with Chile demonstrates that significant effort is made in recent SIAs, including global analysis of economic, social, and environmental sustainability as well as impacts on human rights (including the right to health, to adequate food, and to work), SMEs, women, and rural communities EC (2019).The accompanying CP included opinion surveys, consultation dialogues and workshops with stakeholders in both the EU and in Chile, spanning a crosssection of CSOs.The SIA report concluded that any effects of the envisaged modernization of the agreement would have only minor, if any, implications for the various rights and groups considered, reflecting the limited nature of the envisaged policy changes and the fact that outcomes and possible 12 Zerk (2019) provides a complementary assessment and discussion from a legal perspective.improvement in performance for the areas and issues of interest are a matter of domestic policy and institutions that are only indirectly affected by a trade agreement, if at all.There is no prioritization and little if any focus on how the trade agreement can mobilize action at the domestic level.
In what follows, we use the example of human rights to discuss three interrelated reasons why the treatment of NTPOs in SIAs is not only of limited salience to trade negotiators but also does little to inform decision-makers when considering ratification of a trade agreement.We use the example of human rights not to single out this specific issue but as an illustration of the broader challenge of impact assessment focused on NTPOs.Analyzing the impact of trade policy on human rights is not straightforward given that such rights encompass a broad array of human activity: economic, political, cultural, and social. 13Many of the existing legal, political and cultural institutions that determine the prevailing human right situation are difficult to change, and there is not necessarily a direct link between trade, trade policy changes and human rights outcomes.The level of protection of specific rights in specific countries is determined by domestic institutions.Trade often can and will have little bearing on the direction of change in baseline levels of non-trade outcomes of interest.Implicitly acknowledging this limitation, the second edition of the SIA handbook states: This [human rights] analysis is not intended to pass judgement on the actual human rights situation in a country, nor to decide whether the country is eligible for the conclusion of trade negotiations (EC 2016)   Even with a clear delimitation of topical scope, any analysis of the effects on human rights of a trade agreement is an extremely ambitious analytical undertaking.Requiring an assessment of all the issue areas falling under the broad heading of social, civil and environmental protection -viz.the list of issues included in SIA consultation survey instruments -is impossible.Attempts to do so, no matter how diligently, make it difficult to refute allegations that this is little more than a 'tick the box' exercise.
Matters are compounded by the absence of a systematic, evidence-based, and standardized methodology to analyze the impact of trade agreements on human rights.Many scholars advocate for a comprehensive analysis of human rights issues employing tailor-made human rights impact assessments (HRIAs).14However, a robust HRIA methodology that incorporates trade and trade policy as an instrument has not been developed (Brando et al. 2015).In our view it cannot be developed because trade and trade policy is just one of many potential factors influencing non-trade variables.A much more holistic analysis is called for that encompasses domestic institutions, domestic policy and domestic political economy dynamics that generally determine the non-trade performance of a country (as reflected in baseline indicators) and will affect the potential for desired changes in non-trade outcomes.
Focusing on a prospective trade agreement is akin to looking for lost keys under the lamppost because there is light there. 15ven the wide range of rights and stakeholders implicated, the overall feasibility and value added of such assessments is open to question.Even if attention is limited to a sub-set of human rights topics that have a direct economic dimension -such as food security or workers' rights -that can more easily be included in a quantitative SIA and are more likely to be influenced by trade policy, empirical analysis requires the use of a robust methodological framework.The EC (2016) handbook does not delimit the coverage of human rights issues, nor does it provide analytical frameworks to do so.Instead, the guidelines provided in EC (2015b) recommend using a simple positive, negative, or neutral (+/-/0) score to assess direct and indirect effects of a trade agreement on specific human rights, mostly based on induction-based links between the expected economic outcomes of the trade agreement and different rights.The lack of a robust methodological framework makes the analysis of human rights effects in extant SIAs highly speculative and hypothetical.So far, no SIA has generated quantitative estimations that can provide even a broad understanding of the relative magnitude of the potential effects.
Trade policy affects economic outcomes by influencing relative prices of goods and services, and thus sectoral activity, labor markets, public finances, and income distribution.Trade agreements will inevitably affect one way or another the economic, social, political, and personal human rights of all citizens through these channels.Any effects will be heterogeneous and conditional, depending inter alia on worker and household characteristics.The relevant question then becomes how much an agreement affects specific non-trade outcome variables such as human rights and/or specific target groups, not if the agreement will have non-trade effects.What is required is a quantitative assessment of the potential impacts, instead of a qualitative, subjective dichotomous judgement on the existence and sign of any relationship between trade policy and a particular human right.In general, since most trade agreements have modest economy-wide effects, the potential impacts on non-trade outcomes are likely to be limited.Nevertheless, the potential impacts can be significant for specific economic activities and can also affect heterogenous population groups differentially, conditional on skill-level, occupation, economic activity, gender, region and socio-economic background, among others.The plethora of subtopics within the human rights umbrella, together with the potential impact on many sub-national population groups implies many combinations to be analyzed.Some aspects of human rights can be considered in an economic analytical framework, permitting quantitative analysis, but many others do not.For instance, the analysis of the trade policy effects on income inequality and particular worker types has a well-established theoretical basis and empirical tradition in economics.This is not the case for many human rights -e.g., physical integrity, children rights or minority protection.
The lack of a robust, proven methodology to analyze human right impacts is reflected in extant SIA analyses.Of the 29 SIA final reports surveyed in Rojas-Romagosa ( 2020), none provides a quantitative analysis of the potential impact of trade agreements on any of ten human rights covered in that study: civil and political rights, physical integrity, women rights, human trafficking, right to food security, minority protection, children rights and freedom of religion.Assessment of human rights impacts of a potential trade agreement relied on qualitative and inference-based analysis that is inherently speculative and ad-hoc in nature. 16For example, the SIA on the Environmental Goods Agreement, finds that the main (positive) effect of the agreement on human rights was that: "[s]mall solar lights, or renewable energy projects that bring electricity to rural areas, can allow people to work later, to enjoy leisure activities and to read and write during the night, among other benefits."(Development Solutions, 2016)   Such findings with little or no quantitative backing, are common in most SIAs human rights impact assessments.The general impression is that the specific links between trade and selected human rights topics reflect arbitrary decisions by the SIA consortia with an eye to including non-trade issues in the table of contents of the SIA.They do not provide very (if any) useful information on the potential impact of an agreement on the human rights considered.Although the number of non-trade topics analyzed in each SIA has been increasing over time (Rojas-Romagosa, 2020), the assessment of human rights dimensions is not evidence-based, contrary to what is called for in EC ( 2016), but remains unquantified, anecdotal and of little practical use in policy debates.The main contribution of such analyses is to flag specific human rights related outcome variables that could be affected by an agreement -without any presumption that an agreement is in fact salient to such outcomes -and to identify individuals or specific groups of people that may be affected.However, even such indicative analysis is highly conditional on the interrelationships between economic effects (at the macro, sectoral and household level) and the institutional setting that prevails in a partner country.

Enhancing the SIA consultation process
Encouraging stakeholders to exercise voice without an attempt to identify what concerns or issues are most salient from an economic/trade perspective and most important to a representative cross-section of groups in society reduces the value of the CP.In this section we make several suggestions to improve the consultation process, focusing on addressing some of the methodological weaknesses discussed above -improving representativeness, weighting results to permit rank-ordering of issues and identify priority areas for action, and devoting the limited analytical resources available to quantifying baselines and assessing the potential effect of trade reforms on the identified non-trade priorities -and considering alternative instruments (flanking measures) that may be more effective and efficient in improving the selected non-trade outcomes.The latter is an important potential contribution that can be made by the consultative process as stakeholders will have information based on experience on what types of instruments and interventions are likely to be more effective in improving performance on a given non-trade issue.One of the findings from the survey of practitioner views on EU external trade and cooperation in Fiorini et al. (2021) was that most respondents regarded trade agreementspreferential access to the EU market -as a relatively ineffective instrument to achieve NTPOs.
Respondents ranked a range of external cooperation tools much more highly in terms of effectiveness.
Using the CP to solicit such information would be of value to both sides of a trade negotiation as it would identify complementary measures to incorporate into an eventual agreement.

Revisiting the design of consultation surveys
A survey with closed ended questions can provide useful information on which issues are deemed most important by stakeholders and should be prioritized, as respondents' views on NTPOs can be rank ordered.The current approach in EU SIA consultations utilizes a survey instrument that solicits responses to a range of questions, including non-trade issues and related outcomes.In principle this permits the collection of information on the priorities accorded to a range of sustainability/human rights concerns for specific trading partners.However, in practice it does not because the survey instrument is limited to questions whether respondents believe a trade agreement will impact on a long list of variables contained under the heading of 'social issues, including labor rights', human rights and the environment -e.g., employment, incomes, child labor, social protection, safety at work, health conditions, vocational learning, freedom of religion, freedom of assembly, the right to education, the right to family life, the right to enjoyment of the highest attainable standard of physical and mental health, water quality, climate change, air pollution, biodiversity, etc. 17 The closed ended nature of the survey design and associated scaled responses can be used to rank order concerns expressed across these many areas but because there is no information provided on how or why a trade agreement may affect the various outcome variables the exercise arguably is mis-conceived.What is needed is a set of questions that aim at identifying the non-trade issues that matter most to stakeholders, not what they believe the effects of a trade agreement might be -something on which we can expect most respondents to be rationally ignorant and uninformed (as well as possibly biased).Addressing this problem is in part a task for the SIA to address -by providing information on how trade may impact on those values and NTPOs that can be incorporated into economic analysis.But as argued above it is misconceived to focus primarily on trade and trade policy as a channel of transmission of changes in relative prices or, more speculatively, on corollary investment dynamics.A broader approach is needed that focuses on domestic institutions and political economy.questionnaires is to provide guidance to the EC on how and where to relax the 'blueprint' one-size-fitsall approach reflected in the inclusion of a 'standard' chapter on sustainable development in each trade agreement.Redesigning surveys to include closed ended questions on the specific NTPOs that are of concern to the EU and the appropriate (non-trade) instruments to pursue them could help do so.Doing so would be consistent with one of the action points in EC ( 2018) on improving the implementation of trade and sustainable development chapters in EU trade agreements, i.e.: Based on feedback from all partners, the Commission services will identify, consider and address priorities for each partner country throughout the whole cycle of an FTA, starting with the content-scoping phase of a future agreement.In addition to pursuing the negotiation of a "standard" TSD chapter, this may include further specific issues of relevance for a given partner […] The early identification of priority areas and actions should enable a more efficient implementation of commitments under the agreement and should also provide early information on the key needs in terms of awareness and capacity building among the stakeholders […] [enabling] a more focused use of the various tools throughout the cycle, to step up compliance efforts.(EC, 2018, p. 7, italics added)

Deliberative polling as an element of consultation processes
The results of a closed ended survey that permits rank ordering of views on the priority to be accorded to the NTPOs included in trade agreements would be of value to the EC and counterpart governments in and of itself, significantly enhancing the quality of information provided by SIA surveys.Such re-vamped surveys could also become the basis for deeper, more useful consultations by providing a pool of people representing different groups and organizations that can be sampled to generate a representative group of stakeholders.As noted previously, ensuring the representativeness of participation and inputs provided through the CP is an important question.Another challenge concerns the weighting of expressed views on different issues.
One way to address the problem of clarifying preferences and prioritizing concerns across stakeholders with respect to trade agreements and related non-trade regulation is to use deliberative polling as an element of the CP.This technique is designed to overcome both problems of rational ignorance and bias in stakeholder views when responding to surveys and identify a core set of issues that are deemed to be priorities by many of the groups represented.Deliberative polling brings together a representative group of stakeholders who have expressed concerns about elements of a trade agreement to discuss a subject in small groups facilitated by trained moderators, informed by accessible expert briefing materials that provide balanced information on the range of salient issues, including economic effects and non-economic concerns.The goal is to solicit the group's views -through a poll -on a set of issues raised initially in a first round of consultations or a survey.If the group is statistically representative of the relevant stakeholders, the result of the poll should reflect better the conclusions that would have been attained had the population been more informed and more engaged. 19is approach can be adapted to the CP.A first step would be for the EC consultants tasked with a SIA to develop a survey instrument that includes a set of closed end questions on both trade and non-trade dimensions of the status quo and views on how trade and a possible trade agreement could impact on a broad range of NTPOs, including issues that are not part of the 'standard' trade and sustainable development chapter of EU trade agreements.Responses to questions could use a 5-or 7-point Likert scale to capture the variance in (intensity of) respondents' views, as is done in SIA survey questionnaires, but focus more specifically on soliciting information on perceived priority non-trade issues.
It is often pointed out in the academic literature that establishing whether the CSOs and business associations that participate in consultations are representative of the constituency they claim to speak for is not straightforward.This problem can be attenuated by using the set of respondents to the surveys as the sampling frame for a consultation that takes the form of a deliberative poll.This would help offset the bias that is inherent -and unavoidable -with the approach that is used currently where specialized and self-interested groups are asked to participate in an open-ended consultation process in which they are encouraged and free to provide their views.A representative group to be invited to participate in a deliberative polling process can be selected randomly from a set of survey respondents that indicate a willingness to participate in a follow-on deliberative process in which travel, and related costs are covered by the EC.Representativeness can be attained through stratification, using information provided by respondents on their professional affiliation, geographic location, gender, education, experience, etc. Basing consultations on a representative sample of survey respondents would offset potential selection bias, with stratification ensuring that specific groups of interest or concern are included.
To operationalize this approach the survey response rate should be sufficient to generate a large enough pool to create a sampling frame encompassing the salient groups in society.This can be achieved through a pro-active effort to disseminate and raise awareness of the survey among stakeholders, including in partner countries -e.g., by using contact lists maintained by the relevant EU missions, international development agencies and nongovernmental organizations with a local presence.
Snowballing will help ensure that the sampling frame encompasses a broad range of respondents.
Including international organizations and CSOs with expertise in different non-trade issues, including country-specific knowledge and experience in the survey, will help to address problems of asymmetric information and knowledge.
The advantage of a deliberative poll is that it can be designed to be (more) representative of the population than one that relies on the self-interest of different groups to participate in a CP, with engagement of groups in (large) part determined by how binding their resource constraints are.The voting element of the deliberative process, with participants responding to a poll at the beginning of their deliberations and again at the end after having engaged in an informed, facilitated discussion of the issues can help address the problem that the current CP does not include a mechanism for objectively weighing views and priorities of different groups in society.Such weighting can be achieved by designing the polls to rank order non-trade issues as well as associated potential policy instruments that should be considered to pursue the selected NTPOs, consistent with the aim in EC (2018) noted above to generate early information on the key needs in terms of awareness and capacity building.
A deliberative polling process can also address the problem of 'rational ignorance', another source of bias.For the deliberative polling process to be effective a robust information base is required, including preliminary analysis of the potential results of a trade agreement.The latter can be supplied by the SIA, conditional on the quality and extent of available data and methodological constraints discussed previously regarding assessments of how trade reform may impact on NTPOs.This potential constraint could be addressed in part by synthesizing the experience with implementation of trade agreements and associated ex post evaluations, as well as the academic literature analyzing the relationship between trade and trade reforms on both economic and non-trade issues.The information problem can also be addressed in part through engagement with international organizations with specialized expertise and country presence/knowledge.

Multi-stakeholder value chain-based deliberations
Another, complementary approach is to conceptualize the CP as a multi-stakeholder partnership that supports dialogue and informed deliberation among representatives of the set of actors most concerned with changes in trade regulation.Including business, consumer groups, representatives of labor, and actors with a specific interest in noneconomic issues.Multi-stakeholder initiatives are complex to manage,20 but one possible element of an explicitly deliberative CP would be to pursue a value chain approach that is organized around specific sectors or economic activities (Kolben, 2017; Marx et al.,   2017).This can be an alternative means of identifying a representative set of stakeholders with a common interest in an economic activity that may be affected by trade policy reforms.A value chainbased approach has the advantage of focusing attention on a concrete set of activities as opposed to a more general cross-cutting discussion of trade policy and NTPOs.Organizing deliberations around several value chains that are economically significant would ensure that both upstream (e.g., raw materials, parts, components) and downstream (e.g., distribution) activities are considered, potentially helping to identify elements -including the location, composition of the associated workforce, etc. -of the production processes of goods and services that are most salient from the perspective of specific non-trade issues and NTPOs (Findlay and Hoekman, 2020).In practice trade-related policies will affect a cross-section of firms that are linked together as suppliers or buyers in supply chains and production networks, in contrast to the industry and product-specific focus that tends to be taken in SIAs.A value chain approach to consultation has become more salient given the intention of the EU to develop mandatory standards for internal operation of international supply chains of EU headquartered companies (EC, 2021).

Using the CP to support implementation
The return on the effort that is put into identifying relevant actors and seeking to ensure representativeness in the CP could be increased by using the process to identify complementary instruments and actions to pursue prioritized NTPOs and to suggest baselines and key performance indicators to include in a trade and associated partnership agreement.As noted, most practitioners view trade agreements as relatively ineffective instruments to improve performance on non-trade variables (e.g., Yildirim et al. 2021).
The SIA includes a website to provide information and enhance transparency.This feature of the SIA could become a building block for a more regular interaction between officials, business and other stakeholders by being designed as "knowledge platforms": mechanisms to promote collection, analysis, and diffusion of knowledge and experience.Rather than the EC and the partner government simply periodically consulting with the private sector and civil society, knowledge platforms allow for sustained engagement among stakeholders.Such platforms already have been created by national governments and international organizations for a variety of policy areas. 21For example, the Dutch government has established a platform on electromagnetic fields that brings together academics, regulators, government agencies and CSOs with concerns about the health effects of electromagnetic fields. 22Even if the suggestions made above are not adopted, knowledge platforms can provide a better basis both for ex-ante dialogue on the design and potential effects of a trade agreement as well as for ex-post monitoring and learning about the implementation of an agreement.This could include a focus on establishing baseline indicators on variables and areas of interest, both economic and noneconomic, and consider meaningful performance indicators and metrics that can be used as focal points for governments and stakeholders measure progress -or lack thereof -in realizing the objectives of the trade agreement.This in turn could contribute to monitoring the implementation of policy reforms and progress in achieving NTPOs.The suggestion to create knowledge platforms for EU trade agreements and associated implementation projects can build on already existing local stakeholder fora when they exist, as suggested by Ashraf and Seters (2020) in their analysis of ways the bolster the effectiveness of domestic advisory groups for specific trade agreements.
The suggestions made in the previous sub-section regarding deliberative polling and value chain partnerships can feed into a knowledge platform.These types of initiatives can build on the civil society dialogues in the EU organized by DG Trade and on local stakeholder fora where these already exist, enhancing local ownership. 23Domestic institutions and civil society mechanisms complement ex-ante SIAs and associated consultations as mechanisms to provide inputs and feedback on implementation of NTPOs in trade agreements.Domestic Advisory Groups (DAGs) are an example, conceived as mechanisms intended to further the goals of the trade and sustainable development dimensions of EU trade agreements.Ashraf and Seters (2020) note that DAGs and joint civil society meetings were established with a view to encourage monitoring by civil society of progress in promoting sustainability dimensions of EU trade agreements with partner countries but that the lack of substantial outcomes has made stakeholders increasingly critical of the usefulness of these mechanisms (see also Harrison et al Our recommendations may help do so.The challenge extends beyond ex ante SIAs, including not just design but the process and conduct of negotiations, and the implementation phase with associated monitoring and evaluation.A more rigorous ex-ante impact assessment-cum-consultation process that becomes the basis for regular engagement between interested parties, partner governments and the EU (as well as other international organizations and bilateral economic development agencies) can help improve implementation and outcomes.Sustaining engagement and developing partnerships with stakeholders, including the business community and actors that participate in (depend on) specific international value chains, can do much to generate feedback on progress or lack thereof and identify ways in which to improve results.Steps in this direction would help attain policy coherence objectives and be more effective than the current segmented approach towards SIAs, consultations and implementation-cum-enforcement of treaty provisions.The suggestions made above would help provide a basis for sustained engagement by and with CSOs, governments and economic actors in civil society mechanisms in EU trade agreements.Some of the processes used to form DAGs to ensure representativeness of participants are very similar to what is needed for deliberative polling.Conversely, the SIA approach of focusing on both economic and noneconomic dimensions of a trade agreement would be beneficial to incorporate into the DAGs by going beyond the current focus on sustainability to include as well as a focus on economic dimensions.

Conclusion
It is an inevitable feature of any trade agreement that there will be adjustment costs -this is a necessary condition to realize the overall net gains from trade liberalization.Recognizing this and putting in place credible mechanisms to assist groups that may lose is an important element of an SIA, but SIAs can also do more to identify how a trade agreement can -and cannot -be used to promote NTPOs.This calls for evaluating issues that are not covered in the standard trade computable general equilibrium analytical frameworks used in SIAs.As discussed in Rojas-Romagosa (2020) this can include extensions using environmental and climate assessments (e.g., air pollution, water and land use, energy generation, and environmental taxation, adaptation, and mitigation processes), and micro-simulations that can account for economic and social effects at the household level. 24necessary condition for the quantitative analysis of trade-NTPO linkages in ex ante impact assessments is to narrow down the number of NTPOs to focus on.The revisions to the consultation process suggested in this paper could help to provide both the guidance and the 'head room' needed for better evidence-based quantitative assessments of specific non-trade topics in SIAs.For consultations to be most useful in informing trade negotiations they should serve the purpose of identifying priority nontrade issues for stakeholders in the EU and the partner country and aim at identifying a limited set of NTPOs to be considered in the SIA analysis and the design of a trade agreement and associated external cooperation activities.The current wide ranging but shallow treatment of non-trade issues in SIA survey instruments, which solicit subjective perceptions whether a trade agreement will impact negatively on a large number of social, human rights and environmental outcome variables, does not serve the needs of policymakers seeking to improve non-trade outcomes, or generate the specificity needed for accountability, reducing the credibility of the exercise.
A consultative process that combines a closed ended survey instrument that focuses on the NTPOs that are (or could be) embodied in a trade agreement using a sampling frame that spans the groups in society of interest/concern to policymakers with a deliberative process that includes a polling mechanism to identify (rank-order) priorities and associated policy instruments would help ensure both the representativeness of the CP and deliver more useful inputs and guidance for negotiators and the analysts charged with implementing the SIA.By identifying a limited set of priority non-trade issues areas, the SIA can devote attention to a more in-depth and evidence-based (quantitative) impact assessment of these issues.Doing so would help address the weakness of indirect and induction-based analyses that in many cases are highly speculative and provide very limited information on how to design a trade agreement to realize non-trade goals.In parallel, such analysis should include a focus on alternative policy instruments as well, given that trade policy is unlikely to be a very effective tool to achieve NTPOs (Harrison et al. 2019; Borchert et al. 2021; Ferrari et al. 2021; Yildirim et al. 2021).Final 24 For example, detailed poverty impacts (by groups and regions), different dimensions of inequality (income, consumption, factor incomes, regional disparities, gender gaps), and information on specific groups (minorities, migrants, and other vulnerable groups) and across different types of workers (by skills, occupation, economic activity, and level of formality) can be used for analysis of possible impacts of a trade agreement on a range on NTPOs.
For instance, the Transatlantic Trade and Investment Partnership (TTIP) between the EU and the USA was launched in June 2013 with the first negotiation round held on 7 July 2013.The tender for the contract to provide the trade SIA was only published on 24 July 2013.The negotiations ended without a conclusion at the end of 2016, before the final SIA report was issued (in March 2017).Similarly, the EU-Canada Comprehensive Economic and Trade Agreement (CETA) was launched in May 2009 and the first round of negotiations occurred on 19 October 2009.The tender for the SIA was posted in January 2010 and the final SIA report was published in June 2011.
Given the inherent limits to what can be done in focusing on how trade might affect non-trade issues, a complementary process of deliberative polling may help to generate more informed results from consulting with stakeholders.The scaled closed ended survey design is not used in the (ongoing) open public consultation on the Trade and Sustainable Development Review, which is limited to a set of openended questions. 18Arguably is should have been.One benefit of going beyond open-ended ., 21 E.g., the FAO Sustainable Food Value Chains Knowledge Platform (www.fao.org/sustainable-food-valuechains/home/en/); the OECD Initiative for Policy Dialogue on Global Value Chains, Production Transformation and Development (www.oecd.org/dev/GVC_Initiative_Brochure_2015-01.pdf); the World Bank's Global Partnership for Social Accountability Knowledge Platform (https://gpsaknowledge.org/); and the UN Global Compact on Supply Chain Sustainability (www.unglobalcompact.org/engage-locally/manage/engagement/supply-chain-sustainability). 22Knowledge Platform on Electromagnetic Fields and Health, at www.kennisplatform.nl/English/knowlegdeplatform.aspx. 23See https://trade.ec.europa.eu/dialogue/.20 2019).It is important that DAGs show results that justify the investment made towards them by stakeholder groups in promoting sustainable development outcomes.