• It allows easy customization and exchange of rules via SRX files
• It preserves whitespace surrounding break points
• It offers advanced XML/HTML tag handling: it won't be fooled by false breaks

Some other advantages that are not unique to SRX:

• It is offered under a very permissive license
• It is relatively lightweight as a dependency
• It is fast (though this depends somewhat on the ruleset you use)

• It is inherently rule-based, with all of the weaknesses that implies
• It is not very accurate on the Golden Rules

Caveats

The SRX spec calls for ICU regular expressions, but this library uses standard Ruby regexp. Please note:

• Not all ICU syntax is supported
• For supported syntax, in some cases the meaning of a regex may differ when

• The following ICU syntax is supported through translation to Ruby syntax:

• \x{hhhh} → \u{hhhh}

• \0ooo → \u{hhhh}

Installation

Add this line to your application's Gemfile:

gem 'srx'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install srx

Usage

Use the default rules like so. Specify the language according the <maprules> of your SRX (usually two-letter ISO 639-1 codes).

require 'srx'

data = Srx::Data.default
engine = Srx::Engine.new(data)
engine.segment('Hi. How are you?', language: 'en') #=> ["Hi.", " How are you?"]

Or bring your own rules:

data = Srx::Data.from_file(path: 'path/to/my/rules.srx')
engine = Srx::Engine.new(data)

Specify the format as :xml or :html to benefit from special handling of tags:

# This should only be one segment, but handling as plain text incorrectly
# produces two segments.
input = 'foo <bar baz="a. b."> bazinga'

Srx::Engine.new(Data.default).segment(input, language: 'en')
#=> ["foo <bar baz=\"a.", " b.\"> bazinga"]

Srx::Engine.new(Data.default, format: :xml).segment(input, language: 'en')
#=> ["foo <bar baz=\"a. b.\"> bazinga"]

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/amake/srx.

License

The gem is available as open source under the terms of the MIT License.