%% An abstract: %% %% Exception safety is a difficult topic in any language. In non-GC'd languages %% like C++, it can be very hard to write exception-safe code. In Ruby, it's a %% little easier, but there are still quite a few "gotchas". %% %% The goal of this talk is to discuss some of the exception safety issues that %% show up on the mailing list, to propose some solutions, and to discuss %% problems not yet solved. If time permits, exception-safety in Ruby extensions %% written in C and C++ will also be discussed. Some knowledge of C/C++ is %% recommended but not required. %% %% An outline: %% %% I. Exception safety in Ruby %% A. Resource allocation/deallocation %% 1. finalizers - hard to use, doesn't guarantee cleanup %% 2. blocks - guaranteed cleanup, but potentially more expensive %% B. Exception safety guarantees %% 1. basic guarantee - invariants preserved, no resources leaked %% 2. strong guarantee - state remains the same if an exception is thrown %% a. ensure block - useful for rolling-back static operations %% b. undo/rollback - rollback dynamic operations in reverse order %% (see RubyTreasures and MetaRuby) %% 3. no-throw guarantee - necessary when operations cannot be undone %% C. Intercepting exceptions (with ensure) %% a. re-raising the exception doesn't always work %% b. using ensure to intercept the exception means the exception can be %% propogated without being re-raised. %% E. Some functions must NOT raise exceptions %% 1. freeze %% a. makes "deep freeze" difficult if it can raise exceptions (since %% "unfreeze" is impossible (note that taint/untaint are okay, since %% they are undoable). %% b. [ruby-talk:40022] %% 2. each %% a. necessary for implementing freeze in some objects %% b. CAN pass exceptions raised in the block (since it is user code) %% 3. <=>, ==, finalizers %% a. Exception will not get propogated nor printed, just ignored %% b. [ruby-talk:40612] %% 4. The * operator %% a. Used for multiple assignment, calls to_a() %% b. If to_ary() fails, then * can fail, so don't use it for swap() %% operations. %% E. Error messages (reference Andrei's talk) %% 1. Good exception message != good error message %% 2. How to write a generic component for generating error messages? %% F. Exceptions in threads %% 1. abort_on_exception - forgetting it is a common mistake %% 2. alternative: catch exception at highest level and log it %% G. Errno exceptions %% 1. Where Ruby 1.6.x returns an error code, %% 2. Ruby 1.7.x raises an Errno::xxx exception. %% %% II. Exception safety in extensions %% A. C - easy; C++ - hard %% B. First try: rb_rescue2 (doesn't work with catch/throw) %% C. Second try: rb_protect (how do I re-throw the symbol?) %% 1. This technique used by Exceptional Ruby (my wrapper for writing %% exception-safe C++ extensions) %% D. Alternative: allocate everything on the heap and register it with the %% Ruby GC %% %% III. Resources %% A. http://rm-f.net/~cout/ruby/treasures/RubyTreasures-0.4/lib/exc/ %% B. http://www.gotw.ca/publications/xc++.htm %% C. http://www.boost.org/more/generic_exception_safety.html %% D. http://www.metabyte.com/~fbp/stl/eh_contract.html %% %% Default fonts %deffont "standard" xfont "helvetica-medium-r", tfont "arial.ttf", tmfont "arial.ttf" %deffont "thick" xfont "helvetica-bold-r", tfont "arialb.ttf", tmfont "arialb.ttf" %deffont "typewriter" xfont "fixed-medium-r", tfont "cour.ttf", tmfont "cour.ttf" %% %% Default settings per each line number %default 1 area 90 90, leftfill, size 2, fore "yellow", back "black", font "thick", bgrad %default 2 size 6, vgap 10, prefix " ", ccolor "black" %default 3 size 2, bar "gray70", vgap 10 %default 4 size 5, fore "white", vgap 30, prefix " ", font "standard" %% %% Default settings for tab-indented lines %tab 1 size 5, vgap 40, prefix " ", icon box "green" 50 %tab 2 size 4, vgap 40, prefix " ", icon arc "yellow" 50 %tab 3 size 3, vgap 40, prefix " ", icon delta3 "white" 40 %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page %center Exceptional Ruby Ruby Conference 2002 Paul Brannan paul@atdesk.com %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Overview Exceptions happen We need to deal with them correctly A program shouldn't crash just because it gets an exception A program should never have undefined behavior A program should never leak resources GC makes the job easier, but doesn't solve it. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A Simple Example %% %% %font "typewriter", size 3 1 def read_configuration(filename) 2 f = File.open(filename) 3 host = f.gets 4 port = f.gets.to_i 5 f.close 6 return host, port 7 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page One Solution %% What happens if f.close fails? %font "typewriter", size 3 1 def read_configuration(filename) 2 f = File.open(filename) 3 begin 4 host = f.gets 5 port = f.gets.to_i 6 ensure 7 f.close 8 end 9 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page The Right Way %font "typewriter", size 3 1 def read_configuration(filename) 2 File.open(filename) do |f| 3 server = f.gets 4 port = f.gets.to_i 5 return host, port 6 end 7 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Basics of Exception Safety Exception Safety Guarantees (Dave Abrahams) basic guarantee invariants preserved (the object remains in a consistent state) no resources leaked %pause strong guarantee state remains the same if an exception is thrown %pause no-throw guarantee necessary for proper cleanup when exceptions are thrown %pause exception-safe code meets the basic exception-safety guarantee exception-neutral code passes all exceptions to the caller purely functional code is always exception-safe %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Back to the example %font "typewriter", size 3 1 def read_configuration(filename) 2 f = File.open(filename) 3 host = f.gets 4 port = f.gets.to_i 5 f.close 6 return host, port 7 end %font "standard" Question: Is this exception-safe? %pause If an exception is raised an exception, then: The state is consistent No resources are leaked Answer: Yes, but... %% (the resources are not leaked, but they are not immediately freed) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Resource allocation/deallocation %font "typewriter", size 3 1 # Method 1: Use finalizers (let the GC do the work) 2 class Foo 3 class Helper 4 def initialize 5 @x = allocate_some_resource 6 end 7 8 def finalize 9 free_some_resource(@x) 10 end 11 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Resource allocation/deallocation (cont'd) %font "typewriter", size 3 12 def initialize 13 @helper = Helper.new 14 ObjectSpace.define_finalizer( 15 self, 16 self.class.finalizer(@helper)) 17 end 18 19 def self.finalizer(helper) 20 return proc { helper.finalize } 21 end 22 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Resource allocation/deallocation (cont'd) %% TODO: Can I find a better name for "self.with"? %font "typewriter", size 3 1 # Method 2: Use blocks (like C++''s RAII) 2 class Foo 3 def initialize() 4 @x = allocate_some_resource 5 end 6 7 def finalize() 8 deallocate_some_resource(@x) 9 end 10 11 def self.with 12 foo = Foo.new 13 begin 14 yield 15 ensure 16 foo.finalize 17 end 18 end 19 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Transactions %font "typewriter", size 3 1 require 'pstore' 2 3 p = PStore.new('foo.db') 4 p.transaction do 5 p['foo'] = 42 6 p['bar'] = foo() 7 end %font "standard" %pause Transactions allow your code to do work in memory then commit only when the work is done. Whenever possible, write methods that only modify state when all other work is done. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Rolling back changes Consider the following: %font "typewriter", size 3 1 class UserDB 2 def adduser(username, password, address, phone) 3 @phone_db.adduser(username, address, phone) 4 @system1_db.adduser(username, password) 5 @system2_db.adduser(username, password) 6 end 7 end %font "standard" Is this exception-safe? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Rolling back changes (cont'd) The snippet does not meet the strong guarantee %pause Does it meet the basic guarantee? %pause We need to rollback changes if any step fails %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Rolling back changes (cont'd) Solution: %font "typewriter", size 3 1 class UserDB 2 def adduser(username, password, address, phone) 3 rollback = [] 4 begin 5 @phone_db.adduser(username, address, phone) 6 rollback.push proc { @phone_db.deluser(username) } 7 @system1_db.adduser(username, password) 8 rollback.push proc { @system1_db.deluser(username) } 9 @system2_db.adduser(username, password) 10 rollback.clear 11 ensure 12 rollback.reverse_each { |p| p.call } 13 end 14 end 15 end %font "standard" %% What would happen if deluser fails? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page The no-throw guarantee Question: What happens if deluser fails? %pause Question: What could we do if there were no deluser method? %pause Common methods that should not throw: finalizers ensure blocks freeze (makes "deep freeze" difficult) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Exception-safety in extensions Three problems: Correct behavior in presence of exceptions Ruby exceptions should not enter C++ code C++ exceptions should not enter Ruby code %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Calling C++ code from Ruby Can't let the C++ exception leave C++ code Wrap all C++ functions with try/catch blocks Use Swig's %except directive %% TODO: example? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Calling Ruby methods from C++ Ruby exceptions and C++ exceptions don't mix Need to ensure that resources are not leaked Register everything with the GC (swig) Translate Ruby exceptions into C++ exceptions and back %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page An example %font "typewriter", size 3 1 VALUE foo(VALUE self, VALUE x) { 2 Foo f; 3 f.foo(NUM2INT(x)); 4 return Qnil; 5 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution %font "typewriter", size 3 1 struct Ruby_Jump_Tag { 2 Ruby_Jump_Tag(int t) : tag(t) { } 3 int tag; 4 }; 5 6 struct Ruby_Exception { 7 Ruby_Exception(VALUE e) : ex(e) { } 8 VALUE ex; 9 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 typedef VALUE (*RUBY_VALUE_FUNC)(VALUE); 2 VALUE rb_cpp_protect(RUBY_VALUE_FUNC f, VALUE arg) { 3 int state = 0; 4 VALUE retval = rb_protect(f, arg, &state); 5 if(state != 0) { 6 throw Ruby_Jump_Tag(state); 7 } 8 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 #define RUBY_TRY \ 2 int ruby_jump_tag = 0; \ 3 VALUE ruby_exc = Qnil; \ 4 goto start_of_RUBY_TRY; \ 5 ruby_exception: rb_exc_raise(ruby_exc); \ 6 ruby_jump_tag: rb_jump_tag(ruby_jump_tag); \ 7 start_of_RUBY_TRY: \ 8 try %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 #define RUBY_CATCH \ 2 catch(Ruby_Jump_Tag const & ex) { \ 3 ruby_jump_tag = ex.tag; \ 4 goto ruby_jump_tag; \ 5 } \ 6 catch(...) { \ 7 RUBY_RETHROW(rb_exc_new2( \ 8 rb_eRuntimeError, \ 9 "unknown C++ exception thrown")); \ 10 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 #define RUBY_RETHROW(ex) \ 2 ruby_exc = ex; \ 3 goto ruby_exception; %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 VALUE rb_cpp_num2long(VALUE x) { 2 return (VALUE)(NUM2LONG(x)); 3 } 4 5 unsigned long CPP_NUM2INT(VALUE x) 6 { 7 return (unsigned long)(rb_cpp_protect( 8 Conversion_Helpers::rb_cpp_num2ulong, x)); 9 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A generic solution (cont'd) %font "typewriter", size 3 1 VALUE foo(VALUE self, VALUE x) { 2 RUBY_TRY { 3 Foo f; 4 f.foo(CPP_NUM2INT(x)); 5 return Qnil; 6 } catch(std::bad_alloc & ex) { 7 RUBY_RETHROW(rb_exc_new2( 8 rb_eRuntimeError, 9 "failed to allocate memory")); 10 } RUBY_CATCH 11 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Exceptions with Swig %font "typewriter", size 3 1 void foo(int x); 2 3 %except(ruby) { 4 RUBY_TRY { 5 $function 6 } catch(std::bad_alloc & ex) { 7 RUBY_RETHROW(rb_exc_new2( 8 rb_eRuntimeError, 9 "failed to allocate memory")); 10 } RUBY_CATCH 11 } %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Is this code exception-safe? %font "typewriter", size 3 1 class Foo 2 def initialize 3 @x = Bar.new 4 @y = 10 5 @z = 20 6 end 7 8 def foo 9 x = @x 10 @x = @y, @z 11 @y, @z = x 12 end 13 end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Answer: probably %font "typewriter", size 3 1 class Bar 2 def to_ary 3 raise "uh oh!" 4 end 5 end 6 7 f = Foo.new 8 begin; f.foo; rescue; end %font "standard" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Summary Exceptions are hiding everywhere in Ruby Writing code that works correctly in the presence of exceptions is not easy, but is possible Some important guidelines: Use ensure blocks whenever possible Don't allow ensure blocks to raise new exceptions Write methods that only modify state when all other work is done. Exception-safety is not without overhead Exception-safety in extensions can be partly solved by translating exceptions %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Resources %size 4 %icon box "green" 50 Exceptional C++ by Herb Sutter %icon box "green" 50 http://www.gotw.ca/publications/xc++.htm %icon box "green" 50 http://www.boost.org/more/generic_exception_safety.html %icon box "green" 50 http://www.metabyte.com/~fbp/stl/eh_contract.html %icon box "green" 50 http://rm-f.net/~cout/code/ruby/treasures/RubyTreasures-0.4/lib/exc/