twilio-ruby is taking too much memory.

[mhammiche]

Version: 5.8

twilio-ruby is taking too much memory.

Steps to Reproduce

I audited my project dependencies with the help of derailed_benchmarks and I was surprised too find twilio-ruby at the top of mem usage even before the rails stack.

Here is an extract of the derailed gem

$  bundle exec derailed bundle:mem

TOP: 89.0391 MiB
  twilio-ruby: 28.1602 MiB
    ~/.rvm/gems/ruby-2.3.1/gems/twilio-ruby-5.8.0/lib/twilio-ruby/rest/ip_messaging/v1/service.rb: 0.3047 MiB
    ~/.rvm/gems/ruby-2.3.1/gems/twilio-ruby-5.8.0/lib/twilio-ruby/rest/api/v2010/account.rb: 0.3008 MiB
  rails/all: 20.332 MiB
    rails: 9.3125 MiB (Also required by: active_record/railtie, active_model/railtie, and 8 others)
...

[philnash]

This is interesting. Thanks for bringing it up. Can I ask how you are using twilio-ruby in your application? I found that just including it in the Gemfile lead to 17MiB in use (which is still a lot compared to other gems) but I wonder how to replicate that 28MiB that you're seeing.

Thanks

[philnash]

Could you test version 5.9 and see if it’s any better?

[mhammiche]

I just checked version 5.9. It does not make it better. derailed_benchmark still give me about 28 MB.
I understand the rest client is huge. It is about a hundred of ruby file each taking 0.15 MB

But we are only using it to send small text message like a confirmation code. May be the best solution for our use case is to call the api directly without relying on twilio ruby

[philnash]

I understand, if you're only using it for one message sending, then the whole library and this memory footprint is probably a bit much.

In my testing I only found that derailed_benchmark showed it using ~16-17MiB, can you share how you are using the library within your application?

You've got me thinking about how this could be modularised better so that you could only include the bits that you need, but that is going to take some work. If all you are doing is sending messages and this memory is important, then for now I recommend you implement a call to the API directly. Let me know if I can help with that at all.

[skplunkerin]

Even ~16-17MiB is still really high... twilio-ruby is my 2nd highest Gem in my codebase, at 12.6797 MiB (with my 3rd highest being 5.1367MiB).
I'm using this Gem very minimally (to send an SMS), so using this much MiB is really crazy. Is there a way to decrease this further? Or to only require the parts of this Gem that are needed? (i.e. for using @client.messages.create( message ))

I'm using twilio-ruby v5.10.3

[philnash]

Sorry to take an age to respond to this.

I've had another idea, which is to try to get the library to lazy require the resources it needs as it uses them, rather than loading them all up front. That would reduce initial memory usage, particularly in the case that you are only using it to send a message. I will try to get some time to experiment with this soon.

I would also suggest that if you are only sending messages, then perhaps using a plain HTTP client and building the request yourself might be easier. Using the full library is great if you are doing a lot of stuff with Twilio, but if @client.messages.create is all you are doing then you can cut the amount of code by just cutting the library itself.

[kevinelliott]

@philnash Have you had a chance to do your experiments? We'd really benefit from the reduction in memory at boot. This is also our second largest gem (behind rails itself), which feels incredibly abusive for an API client (and we have many).

[philnash]

I have not had the opportunity to do so yet, no.

[kevinelliott]

:(

[philnash]

I had a quick look last night after leaving that message and I do believe that the approach of lazy loading modules as they are used will help. It's going to be a big job to write that into the framework though. I'll try to put a proof of concept forward then look at porting it into the library generator that we use internally. I can't promise anything is going to happen soon though as this is a big change to the way the library is written.

[kevinelliott]

This would significantly help out the footprint of many Rails stacks! Thank you.