Web Appendix

This appendix provides two alternative strategies to identify misclassified agreements. These are used as robustness checks on the method and results presented in the text of the paper. First, instead of using the independent variables presented in the paper separately (the relevant variables are uncertainty about behavior, enforcement problems, commitment problems, and whether an agreement is multilateral), these variables can be collated into a single, ordinal variable that captures the sheer number of cooperation problems. To facilitate the exposition, I will refer to a multilateral agreement as an additional “cooperation problem” in the following. Second, this ordinal variable can be disaggregated into dummy variables that capture the respective number of cooperation problems. Notably, both of these strategies are more restrictive than the model employed in the text of the paper since these alternative strategies impose the assumption that all of the independent variables have identical effects on the presence of formal punishment provisions. The results presented in Table 1 clearly defy this assumption. As an additional robustness check, this appendix also presents the results from the omission of individual independent variables.

Total Number of Cooperation Problems

For the first model, a variable was created by coding each cooperation problem as either 0 or 1, depending on whether it was present or not in the agreement. These are then added together to create a new aggregate independent variable for each observation. Thus, the variable can range from 0, when an agreement is bilateral and not characterized by uncertainty about behavior, a commitment problem, or an enforcement problem, to 4, when it is multilateral and is characterized by all three cooperation problems. The results below do not change much if uncertainty about behavior, which is the only variable that had a small and statistically insignificant effect in the original model, is omitted.

Unlike the model in the main paper, this model assumes that cooperation problems are interchangeable: enforcement problems have the same effect on the inclusion of punishment provisions as uncertainty about behavior. The model also assumes that moving from none of these cooperation problems to one of them has the same marginal effect as moving from two of these cooperation problems to three of them.

Table A1 presents the results from a regression similar to the one in Table 2 in the main paper, where the dependent variable is coded one whenever an agreement contains formal punishment provisions. As expected, the total number of cooperation problems has a large, positive effect on the probability that formal punishment provisions are included.

Table A2 reports the agreements that are misclassified, and in bold those agreements that were misclassified originally as well. As can be seen, even with this very restrictive model of punishment provisions, there is a large overlap, especially for those agreements with the highest predicted probabilities.

The bold agreements (to reiterate, those that have been misclassified both in the original model and in the model presented in this appendix) constitute a restricted set of agreements with potentially informal punishment. These agreements tend also be more heterogeneous than the remaining agreements, which corroborates the previous results.

Table A3 presents the results using the three heterogeneity measures used in the paper. As can be seen from the table, it is no longer the case that misclassified agreements are significantly more heterogeneous than agreements that were correctly predicted to have punishment provisions. In fact, there is no statistically significant difference among these two sets of agreements.

Table A4, finally, presents the results from the probit regression that includes differences in military capabilities and in economic power. These results are consistent with the results in the main text.

Table A1: Probit Results
Dependent Variable: Presence of Formal Punishment Provisions
Coefficient / Std. Error
Total number of Cooperation Problems / 1.040*** / 0.000
economics / 1.362*** / 0.000
environment / -1.375*** / 0.003
human rights / 0.258 / 0.512
constant / -2.393*** / 0.000
Number Obs. / 234
Log Pseudo-Likelihood / -87.663
Probit estimation, robust standard errors. *** significant at 1%.
Table A2: Misclassified Agreements, Number of Cooperation Problems
Agreement Name / COIL ID / UNTS / Pred. Prob.
Agreement concerning the disposition of certain accounts in Thailand under Article 16 of the Treaty of Peace with Japan of 8 September 1951. / FN 14 / 2913 / 0.504
Exchange of notes (with annex) constituting an agreement regarding the changes which the Government of the United Kingdom propose to introduce in their production and trade policies relating to cereals. / AC 58 / 7450 / 0.504
Austria and Turkey: Agreement regarding trade of agricultural products / AC 2-6 / 30409 / 0.504
Agreement concerning the collection of bills, drafts, etc. / FN 35 / 8851 / 0.504
Exchange of letters constituting an agreement relating to investment guaranties. / IN 16 / 6621 / 0.504
Austria and Bulgaria: Exchange of letters constituting an agreement concerning agricultural products (with annexes).Geneva, 29 March 1993 / AC 2-5 / 30630 / 0.504
Reciprocal Trade Agreement between the United States of America and Mexico / FN 7 / 81 / 0.504
Agreement between Denmark, Finland, Iceland, Norway, and Sweden on the establishment of the Nordic Environmental Finance Corporation (NEFCO) / FN 2-5 / 36248 / 0.504
Exchange of notes constituting an agreement concerning trade in cotton textiles (with annex). / AC 29 / 9770 / 0.504
Austria and Hungary: Exchange of letters constituting an agreement concerning agricultural products (with annexes) / AC 2-4 / 30629 / 0.504
Convention on Nature Protection and Wild Life Preservation in the Western Hemisphere. / EN 48 / 485 / 0.652
Treaty on Collective Security / SE 2-18 / 32307 / 0.766
Convention for the Protection of Human Rights and Fundamental Freedoms. / HR 10 / 2889 / 0.837
International Convention on the Protection of the Rights of All Migrant Workers and Members of their Families / HR 2-5 / 39481 / 0.837
Convention on the Elimination of All Forms of Discrimination against Women. / HR 11 / 20378 / 0.837
International Covenant on Civil and Political Rights. / HR 8 / 14668 / 0.837
United Nations Convention on Independent Guarantees and Stand-by Letters of Credit / FN 2-13 / 38030 / 0.853
Agreement with respect to quality wheat. / AC 57 / 6389 / 0.853
International Convention for the Suppression of the Financing of Terrorism (with annex) / FN 2-17 / 38349 / 0.853
Agreements without punishments, yet predicted to include punishments with probability of at least 0.5.
Table A3: Heterogeneity
Obs. / Mean / Std. Error
Vanhanen
misclassified / 9 / 47.15 / 12.82
all other / 40 / 54.15 / 4.105
total / 49 / 52.87 / 4.04
difference / 7.00 / 10.51
Freedom House
misclassified / 9 / 45.22 / 11.48
all other / 43 / 53.47 / 4.04
total / 52 / 52.04 / 3.86
difference / 8.25 / 10.24
Polity
misclassified / 9 / 10.44 / 3.09
all other / 41 / 12.39 / 1.18
total / 50 / 12.04 / 1.11
difference / 1.95 / 2.90
Table A4: Probit Results
Dependent Variable: Coded 1 if Misclassified Agreement, 0 otherwise.
Coefficient / Std. Error
sd (Military Capabilities) / 85.41*** / 23.66
sd (GDP) / -2.903** / 1.230
constant / -.031 / .734
N / 32
Log Pseudo-Likelihood / -9.149
Coefficients estimates and heteroskedasticity-consistent standard errors. Sample: Agreements in COIL that are either misclassified or that are correctly predicted to have punishment provisions.
***significant at 1%, ** significant at 5%, * significant at 10%.

Dummies for Number of Cooperation Problems

The second alternative model of punishment provisions exploits the total number of relevant cooperation problems calculated above and disaggregates them into a series of dummy variables. For example, a dummy that is 1 if the number of cooperation problems is 1 and is 0 otherwise, another dummy that is 1 if the number of cooperation problems is 2 and is 0 otherwise etc. There are thus 5 dummies that serve as independent variables. While still imposing the assumption that cooperation problems are interchangeable with respect to their effects on the presence of formal punishment provisions, this model allows the marginal effect of two cooperation problems to be different from the marginal effect of three cooperation problems etc. In the regressions, zero cooperation problems are the omitted category such that the coefficients on the dummy variables can be interpreted relative to this baseline category.

Table A5 reports the results from a probit regression. Again, the model performs as expected. The more cooperation problems there are, the higher is the probability that a formal punishment provision is included. The results also suggest that the marginal effects are decreasing in size (and are statistically significantly different from each other), invalidating the assumption of constant marginal effects entertained in the previous model.

Table A6 displays the list of misclassified agreements. As before, agreements that had been misclassified using the original model as well are in bold. The overlap is substantial, and in particular many of the human rights agreements are consistently identified as misclassified.

As before, while the heterogeneity result no longer holds, the result that differences in GDP and military capabilities are related to misclassification is corroborated.

Table A5: Probit Results
Dependent Variable: Presence of Formal Punishment Provisions
Coefficient / Std. Error
one problem / 1.513*** / 0.327
two problems / 2.441*** / 0.32
three problems / 3.127*** / 0.448
four problems / 3.562*** / 0.621
economics / 1.461*** / 0.312
environment / -1.041** / 0.448
human rights / 0.37 / 0.366
constant / -2.738*** / 0.375
Number Obs. / 234
Log Pseudo-Likelihood / -85.638
Probit estimation, heteroskedasticity-consistent standard errors. *** significant at 1%, ** significant at 5%.
Table A6: Misclassified Agreements, Dummies for Number of Cooperation Problems
Agreement Name / COIL ID / UNTS / Pred. Prob.
Convention (with Final Protocol) concerning the reciprocal grant of assistance to distressed persons. / HR 25 / 2647 / 0.529
International Convention on the Suppression and Punishment of the Crime of Apartheid. / HR 22 / 14861 / 0.529
Protocol relating to refugee seamen. / HR 18 / 13928 / 0.529
Memorandum of understanding between the Ministry of Interior of the Republic of Turkey and the Ministry of Internal Affairs of the Republic of Belarus on cooperation in the field of combating trafficking in human beings and illegal migration / HR 2-7 / 44197 / 0.529
Austria and Bulgaria: Exchange of letters constituting an agreement concerning agricultural products (with annexes).Geneva, 29 March 1993 / AC 2-5 / 30630 / 0.593
Exchange of notes constituting an agreement concerning trade in cotton textiles (with annex). / AC 29 / 9770 / 0.593
Reciprocal Trade Agreement between the United States of America and Mexico / FN 7 / 81 / 0.593
Agreement concerning the collection of bills, drafts, etc. / FN 35 / 8851 / 0.593
Agreement concerning the disposition of certain accounts in Thailand under Article 16 of the Treaty of Peace with Japan of 8 September 1951. / FN 14 / 2913 / 0.593
Exchange of notes (with annex) constituting an agreement regarding the changes which the Government of the United Kingdom propose to introduce in their production and trade policies relating to cereals. / AC 58 / 7450 / 0.593
Austria and Turkey: Agreement regarding trade of agricultural products / AC 2-6 / 30409 / 0.593
Austria and Hungary: Exchange of letters constituting an agreement concerning agricultural products (with annexes) / AC 2-4 / 30629 / 0.593
Exchange of letters constituting an agreement relating to investment guaranties. / IN 16 / 6621 / 0.593
Agreement between Denmark, Finland, Iceland, Norway, and Sweden on the establishment of the Nordic Environmental Finance Corporation (NEFCO) / FN 2-5 / 36248 / 0.593
Treaty on Collective Security / SE 2-18 / 32307 / 0.651
International Convention on the Protection of the Rights of All Migrant Workers and Members of their Families / HR 2-5 / 39481 / 0.776
Convention for the Protection of Human Rights and Fundamental Freedoms. / HR 10 / 2889 / 0.776
International Covenant on Civil and Political Rights. / HR 8 / 14668 / 0.776
Convention on the Elimination of All Forms of Discrimination against Women. / HR 11 / 20378 / 0.776
International Convention for the Suppression of the Financing of Terrorism (with annex) / FN 2-17 / 38349 / 0.878
Agreement with respect to quality wheat. / AC 57 / 6389 / 0.878
United Nations Convention on Independent Guarantees and Stand-by Letters of Credit / FN 2-13 / 38030 / 0.878
Agreements without punishments, yet predicted to include punishments with probability of at least 0.5.
Table A7: Heterogeneity
Obs. / Mean / Std. Error
Vanhanen
misclassified / 11 / 48.96 / 10.87
all other / 41 / 54.91 / 4.07
total / 52 / 53.65 / 3.91
difference / 5.95 / 9.63
Freedom House
misclassified / 12 / 49.49 / 8.94
all other / 44 / 54.20 / 4.01
total / 56 / 53.17 / 3.66
difference / 4.81 / 8.98
Polity
misclassified / 12 / 12.25 / 2.50
all other / 42 / 12.55 / 1.17
total / 54 / 12.48 / 1.05
difference / .298 / 2.55
Table A8: Probit Results
Dependent Variable: Coded 1 if Misclassified Agreement, 0 otherwise.
Coefficient / Std. Error
sd (Military Capabilities) / 81.59*** / 22.99
sd (GDP) / -3.367*** / 1.301
constant / .280 / .651
N / 34
Log Pseudo-Likelihood / -10.389
Coefficients estimates and heteroskedasticity-consistent standard errors. Sample: Agreements in COIL that are either misclassified or that are correctly predicted to have punishment provisions.
***significant at 1%, ** significant at 5%, * significant at 10%.

Omitting Independent Variables