group_id	message_id	message_type	origin	original_text	clean_text	annotation_type	annotation_target	annotation_additional	team_performance	performance_change	sol_tracker_message	sol_tracker_all
bd665828-8da1-403d-bf3e-06b46f940cfb	-1	INITIAL	SYSTEM	Butterfly,Ox,Llama,Panda,SYSTEM&&G,8,A,3					0.6	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	a9123a26-5d94-46e6-b1a1-48baa04b000c	MESSAGE	Ox	hello	hello	None	0	0	0.6	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	62ae329c-4268-4c74-b47c-45ae82cc95d2	MESSAGE	Llama	Hi	Hi	None	0	0	0.6	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	6fa79b29-7dc0-456e-af00-2b5b132af5f7	MESSAGE	Butterfly	I only selected cards that had an even number or a vowel to verify the rule.	I only selected cards that had an even number or <CARD> vowel to verify the rule .	Non-probing-deliberation	Solution	complete_solution	0.65	0.05	A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	3f2be4dd-8811-421a-bf4b-d1adf76b5c85	MESSAGE	Panda	must choose the card with vowel to disprove rule	must choose the card with vowel to disprove rule	Non-probing-deliberation	Reasoning	partial_solution	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ea2bead2-3411-4ab9-9b11-ffdf9a23f9fb	MESSAGE	Panda	and conversely must choose card with ODD number to disprove rule	and conversely must choose card with ODD number to disprove rule	Non-probing-deliberation	Reasoning	partial_solution	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	40bf3d91-55ec-4c86-bc63-0d6aea558e27	MESSAGE	Ox	The only card necessary to flip is the one with the A on it.  The rule only suggests that vowels must have even numbers behind them, not that even numbers must have vowels behind them.	The only card necessary to flip is the one with the <CARD> on it . The rule only suggests that vowels must have even numbers behind them , not that even numbers must have vowels behind them .	Non-probing-deliberation	Reasoning	complete_solution	0.65	0.0	A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ca8b00a9-4c96-443a-9f71-95f10e1a7133	MESSAGE	Ox	so there is no reason to check the 8.	so there is no reason to check the <CARD> .	Non-probing-deliberation	Reasoning	complete_solution	0.55	-0.1	8	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	e598965f-3f5a-458a-806a-eed1cfc92994	MESSAGE	Llama	I think we need to choose odd number	I think we need to choose odd number	Non-probing-deliberation	Solution	complete_solution	0.55	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	608b6755-11bf-419a-85ca-7043bc4a06fc	MESSAGE	Butterfly	'wouldn''t you also choose the card with the even number to verify that it has a vowel on the other side of the card?'	'wouldn''t you also choose the card with the even number to verify that it has <CARD> vowel on the other side of the card ?'	Probing	Reasoning	0	0.55	0.0	A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	acf0a0ec-4bb4-4d6a-a150-5e8421e56346	MESSAGE	Panda	but you must choose the 3 to see if there is a consonant or vowel on the other side	but you must choose the <CARD> to see if there is <CARD> consonant or vowel on the other side	Non-probing-deliberation	Solution	partial_solution	0.55	0.0	3,A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ebeeb837-0654-4546-95c3-0c3c8fb64ee7	MESSAGE	Ox	The rules does not suggest even numbers can only have vowels behind them.	The rules does not suggest even numbers can only have vowels behind them .	Non-probing-deliberation	Reasoning	0	0.55	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	7ff04caa-72e5-4752-81c8-5af57b09dd59	MESSAGE	Ox	only that vowels have even numbers behind them	only that vowels have even numbers behind them	Non-probing-deliberation	Reasoning	0	0.55	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	3104bcbb-d3b1-4ce4-90c7-de870b2b05df	MESSAGE	Butterfly	maybe we should choose all cards	maybe we should choose all cards	Non-probing-deliberation	Solution	complete_solution	0.5	-0.05	G,8,A,3	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	be45780d-f2e2-43e8-a487-f812b6a5d6ba	MESSAGE	Panda	must choose vowel to see if the rule is true.	must choose vowel to see if the rule is true .	Non-probing-deliberation	Solution	partial_solution	0.5	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	a3e01066-a4f2-4201-bf5c-a2f88844d0e8	MESSAGE	Panda	must choose an ODD number to see if the rule is true.	must choose an ODD number to see if the rule is true .	Non-probing-deliberation	Solution	partial_solution	0.5	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	74868359-87dd-440a-9ef8-890b1bdccd94	MESSAGE	Ox	i can see where choosing 3 is also appropriate, to see if a vowel is behind that.	i can see where choosing <CARD> is also appropriate , to see if <CARD> vowel is behind that .	Non-probing-deliberation	Reasoning	partial_solution	0.65	0.15	3,A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	aaf80ef3-f2e0-4115-b15f-956a7bf3d9e7	MESSAGE	Ox	to attempt to disprove it	to attempt to disprove it	Non-probing-deliberation	Reasoning	partial_solution	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	62946df1-20fa-4e0f-9f48-4ff0af0259db	MESSAGE	Panda	right. think about it.	right . think about it .	None	0	0	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ef8727c0-caba-4104-a6e4-5e161c76f60c	MESSAGE	Llama	must choose that has odd number in it	must choose that has odd number in it	Non-probing-deliberation	Solution	partial_solution	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	6cff2727-4363-40e4-8a68-4974c3bc991c	SUBMIT	Ox	3,A					0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	d5e780fe-f043-480a-bea1-d535eabb0671	SUBMIT	Ox	3,A					0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	09b4d3cb-b522-4cb0-8620-27a2c90a2870	MESSAGE	Butterfly	must choose cards that display vowel and cards that display even number	must choose cards that display vowel and cards that display even number	Non-probing-deliberation	Solution	partial_solution	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	15032170-5a3c-4896-a2b8-ec7f619289ca	MESSAGE	Llama	8 has vowels in it	<CARD> has vowels in it	Non-probing-deliberation	Solution	0	0.65	0.0	8	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	637a4429-068a-42f6-82c9-59ee58705034	MESSAGE	Panda	ur off the subject.	ur off the subject .	Non-probing-deliberation	Disagree	0	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	0fc923ac-d813-4537-a156-1b456e565335	MESSAGE	Ox	'no, any letter can be behind 8 because there''s no rule for 8 that must be true.'	'no , any letter can be behind <CARD> because there''s no rule for <CARD> that must be true .'	Non-probing-deliberation	Reasoning	0	0.5	-0.15	8	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	e36e0b9c-d40a-43af-8c23-b79f80482445	MESSAGE	Ox	remember that only vowels must have even numbers.  that means that consonants can have even or odd numbers.	remember that only vowels must have even numbers . that means that consonants can have even or odd numbers .	Non-probing-deliberation	Reasoning	0	0.5	0.0		{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["G", "8", "A", "3"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	b5ae7746-350e-4883-839e-95733860e510	MESSAGE	Butterfly	the word eight has vowels but i think the rule is referring to individual letters on one side of a card	the word eight has vowels but i think the rule is referring to individual letters on one side of <CARD> card	Non-probing-deliberation	Reasoning	0	0.55	0.05	A	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	de3e77d2-39a0-4365-ac90-fec2fc5054e7	MESSAGE	Ox	'so it''s ok if its a vowel or consonant behind 8, it wont prove anything.'	'so it''s ok if its <CARD> vowel or consonant behind <CARD> , it wont prove anything .'	Non-probing-deliberation	Reasoning	0	0.6	0.05	8,A	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	7f82a585-59f6-4aa7-961d-54c014385811	MESSAGE	Llama	8 is even number and  behind it is vowels	<CARD> is even number and behind it is vowels	Non-probing-deliberation	Reasoning	0	0.6	0.0	8	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	48177fc3-e0b1-4a1f-b5b5-29b690e6cd7f	MESSAGE	Panda	so choose A and 3 to prove something	so choose <CARD> and <CARD> to prove something	Non-probing-deliberation	Solution	complete_solution	0.6	0.0	3,A	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["8", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	931ac93a-1f08-41a7-9b6e-4d0b7cea433f	MESSAGE	Ox	A and 3 should be correct.	<CARD> and <CARD> should be correct .	Non-probing-deliberation	Solution	complete_solution	0.7	0.1	3,A	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ad60b964-7eb0-477e-b481-f9ab3e4febca	MESSAGE	Llama	why	why	Probing	Reasoning	0	0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	07a0a42d-f6be-46a4-ad5d-e3edb4319824	MESSAGE	Butterfly	the rule does not say tht cards without vowels on one side have an even number on the other. So, cards without vowels could have an even number on the other side	the rule does not say tht cards without vowels on one side have an even number on the other . So , cards without vowels could have an even number on the other side	Non-probing-deliberation	Reasoning	0	0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	ce2092d9-95de-4409-89f0-d78677f6ada6	MESSAGE	Panda	'A and 3 provide proof. the others don''t. try it and see.'	'<CARD> and <CARD> provide proof . the others don''t . try it and see .'	Non-probing-deliberation	Solution	complete_solution	0.7	0.0	3,A	{"Lion": ["G", "3"], "Llama": ["8"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	12d3486b-7a72-4f58-9393-128917ae0781	MESSAGE	Llama	i think is G is correct chose	i think is <CARD> is correct chose	Non-probing-deliberation	Solution	complete_solution	0.7	0.0	G	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	1e34d8ff-878b-40ea-ab33-ee237088ba55	MESSAGE	Ox	'G is not a vowel, there''s nothing to prove with G.'	'<CARD> is not <CARD> vowel , there''s nothing to prove with <CARD> .'	Non-probing-deliberation	Reasoning	0	0.6	-0.1	G,A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["G", "A"], "Panda": ["3", "A"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	9fd3e58d-5a34-462a-b342-21b3e6bf750d	MESSAGE	Panda	'if you flip the G it doesn''t matter what is on the other side. it proves nothing.'	'if you flip the <CARD> it doesn''t matter what is on the other side . it proves nothing .'	Non-probing-deliberation	Reasoning	0	0.45	-0.15	G	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["G", "A"], "Panda": ["G"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	f7fd0680-e687-4708-8016-b8f1531f0fe3	MESSAGE	Ox	'it doesn''t matter if an even or odd number is behind G, it doesn''t let us check the rule.'	'it doesn''t matter if an even or odd number is behind <CARD> , it doesn''t let us check the rule .'	Non-probing-deliberation	Reasoning	0	0.4	-0.05	G	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["G"], "Panda": ["G"], "Butterfly": ["A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	fdcf90c6-e4a1-4a04-8510-b2064de8114d	MESSAGE	Butterfly	so, choose A and 3	so , choose <CARD> and <CARD>	Non-probing-deliberation	Solution	complete_solution	0.45	0.05	3,A	{"Lion": ["G", "3"], "Llama": ["G"], "Ox": ["G"], "Panda": ["G"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	017c2764-b8fd-4353-be72-425a2069a210	SUBMIT	Llama	G,3					0.5	0.05		{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["G"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	4af18e98-1bee-43a2-8185-889c10da71bf	MESSAGE	Panda	A and 3	<CARD> and <CARD>	Non-probing-deliberation	Solution	complete_solution	0.65	0.15	3,A	{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	9fef3938-750b-4d6f-be2c-a0bdcd322d69	MESSAGE	Butterfly	not G and 8	not <CARD> and <CARD>	Non-probing-deliberation	Solution	consider_opposite	0.45	-0.2	G,8	{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["G", "8"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	7b74e7d1-22a8-4244-a3c2-f339e8dc84c2	SUBMIT	Butterfly	3,A					0.65	0.2		{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	56b05664-dc2f-4033-8ad8-21563db3dfbd	SUBMIT	Panda	3,A					0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	cdb892bf-1dc2-4691-bd45-8082c1215a90	MESSAGE	Ox	well good luck everyone.	well good luck everyone .	None	0	0	0.65	0.0		{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["3", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	6ceaf074-f06f-4441-a873-b4980c0668fa	MESSAGE	Butterfly	i will choose A and 3, but not G and 8	i will choose <CARD> and <CARD> , but not <CARD> and <CARD>	Non-probing-deliberation	Solution	complete_solution	0.55	-0.1	3,G,8,A	{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["G"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	d5fdabef-aadd-4c3b-b34b-6215c9d6ba0e	MESSAGE	Ox	A and 3 only.	<CARD> and <CARD> only .	Non-probing-deliberation	Solution	complete_solution	0.7	0.15	3,A	{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	e963b973-be8a-4115-b687-f8538acaed52	MESSAGE	Butterfly	correct, good luck as well	correct , good luck as well	None	0	0	0.7	0.0		{"Lion": ["G", "3"], "Llama": ["G", "3"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	f7f408ee-682f-4727-93d3-1fd2cdaee72e	SUBMIT	Llama	8,A					0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8", "A"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	dd6656f7-6075-4a3b-a131-ef13afff8282	MESSAGE	Llama	good luck	good luck	None	0	0	0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8", "A"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	f452b637-2908-43d7-8310-4240f0345534	SUBMIT	Ox	3,A					0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8", "A"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	4a57a87e-9617-40d6-9e02-f99daf31e61f	SUBMIT	Llama	8,A					0.7	0.0		{"Lion": ["G", "3"], "Llama": ["8", "A"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "G", "8", "A"]}
bd665828-8da1-403d-bf3e-06b46f940cfb	7c311c27-85e8-4f82-a2a3-7760baea3117	SUBMIT	Butterfly	3,A					0.8	0.1		{"Lion": ["G", "3"], "Llama": ["8", "A"], "Ox": ["3", "A"], "Panda": ["3", "A"], "Butterfly": ["3", "A"]}
