1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Drag to adjust the number of frozen columns
Category ( Detailed Blog - https://www.inferless.com/serverless-gpu-market)
Criteria
Banana.dev
Replicate
Beam.cloud
Pipeline
Runpod
Model Import Integration
Basic Functionality
Github
CLI
CLI
CLI
CLI
Types of GPUs offered (Nvidia T4, A100, etc.)
Basic Functionality
A100 only
T4, A100 Both
Nvidia T4s (16G) and A10Gs (24G)
NVIDIA Ampere and Volta GPUs
A100, RTX A6000, RTX 3090/ RTXA5000 , RTX 4000/RTX4500
CPU and memory configurations
Basic Functionality
Not configurable
Not configurable
Configurable
Not configurable
Not configurable
Multi-GPU support
Basic Functionality
No
No
No
No
No
Compatibility with various model frameworks (TensorFlow, PyTorch, ONNX, etc.)
Basic Functionality
TensorFlow, PyTorch, ONNX
TensorFlow, PyTorch, ONNX
TensorFlow, PyTorch, ONNX
ONNX
TensorFlow, PyTorch, ONNX
Custom model support
Basic Functionality
Yes
Yes ( On Request)
Yes
Yes
Yes
GPT- Neo (1.3B) -With Cold Start
Performance
67 secs
249 secs
99 secs
75 secs
290 secs
GPT- Neo (1.3B) - Inference time
Performance
4 secs
4.66 secs
2.1 secs
2.1 secs
3 secs
GPT- Neo (1.3B) - Latency
Performance
71 secs
253.66 secs
101.1 secs
77.1 secs
293 secs
GPT- Neo (1.3B) - Variability
Performance
Highly Variable
Stable
Cold start - Variable
Slightly Variable
Consistent
GPT- Neo (1.3B) - Autoscaling (c=5)
Performance
Does not hold latency
Holds Latency
Does not hold latency
Does not hold latency
Holds Latency
GPT- Neo (125M) - With Cold Start
Performance
32.5 secs
126.63 secs
75.37 secs
32 secs
31 secs
GPT- Neo (125M) - Inference time
Performance
3 secs
3.8 secs
2.5 secs
5.5 secs
1 sec
GPT- Neo (125M) - Latency
Performance
35.5 secs
130.43 secs
77.87 secs
37.5 secs
32 secs
GPT- Neo (125M) - Variability
Performance
Highly Variable
Stable
Cold start - Variable
Slightly Variable
Consistent
GPT- Neo (125M)- Autoscaling (c=5)
Performance
Holds Latency
Holds Latency
Does not hold latency
Holds Latency
Holds Latency
Roberta Large - With Cold Start
Performance
39.7 secs
182.23 secs
93.71 secs
63 secs
101 secs
Roberta Large - Inference time
Performance
1.5 secs
4.6 secs
1.6 secs
3.5 secs
1.63 secs
Roberta Large - Latency
Performance
41.2 secs
186.83 secs
95.31 secs
66.5 secs
102.63 secs
Roberta Large- Variability
Performance
Highly Variable
Stable
Cold start - Variable
Slightly Variable
Consistent
Roberta Large - Autoscaling (c=5)
Performance
Holds Latency
Holds Latency
Does not hold latency
Does not hold latency
Holds Latency
Onboarding process
Ease of use
Good
Okay
Okay
Bad
Good
User interface and experience
Ease of use
Okay
Okay
Okay
Okay
Good
Documentation and tutorials
Ease of use
Good
Good
Okay
Bad
Okay
Pay-per-second, hourly, or monthly billing
Billing
$.00051992 per second (A100 40 GB) No No
Nvidia T4 GPU$0.00055 per secondNvidia A100 GPU$0.0023 per second
$0.00059998per second for inference
0.00055$/sec + 12.99$ platform fee
A100 80Gb = $0.001, 3 other types also available
Free tiers and trial periods
Billing
1 hour of free compute
Free tier available but limit is not mentioned
10 hours of free credit
20$ free credit
None, have to pay and use. You can try emailing for some free credits.
Cost of additional resources (CPU)
Billing
Not offered
CPU$0.0002 per second
$1.75 per GB / mofor storage volume
Not offered
Not offered
Continuous integration and deployment (CI/CD) support
Advanced Features
Yes, Via Github
No, have to push via CLI
Yes, Via Github
No, have to push via CLI
Yes
Model versioning and rollback capabilities
Advanced Features
No
Yes
Yes
Yes
Yes
Metrics and performance monitoring
Advanced Features
Inference Time, Coldstart, Average Coldstart, No of API calls
Inference Time, Average Inference Time, No of API calls
No of API calls, Inference Time
No Of API calls, Latency, Average Latency
No of API calls, Average Latency, GPU/CPU Utilisation
Integration with observability tools (Prometheus, Grafana, etc.)
Advanced Features
No
No
No
No
No
Geographic infra support
Advanced Features
No
No
No
No
Yes
This analysis is created by Inferless.com
34 records

Alert

Lorem ipsum
Okay
View larger version