How to Evaluate a Neocloud: A Practical Guide for AI Infrastructure Teams

By Elemento Team — 2026-08-26

A practical framework for answering which infrastructure can complete your AI workload at the required performance, risk and cost.

A neocloud evaluation should not begin with a provider shortlist.

It should begin with a workload.

That distinction matters because the neocloud market is expanding much faster than most enterprise procurement processes can adapt. Gartner expects worldwide spending on AI-optimized infrastructure as a service to reach $42.3 billion in 2026, up 96.4% from 2025, and $66.1 billion in 2027. Inference alone is forecast to account for $23.3 billion of that spending in 2026, overtaking the $19 billion allocated to training infrastructure.

Gartner also predicts that neocloud providers—cloud platforms purpose-built for AI and high-performance workloads—will capture 20% of a $267 billion AI cloud market by 2030.

Those numbers explain why almost every infrastructure team is now being approached with new GPU offerings, reserved-capacity agreements and promises of better AI economics.

They do not explain which option is right for a particular organization.

The right question is not:

Which neocloud is the best?

It is:

Which infrastructure can complete this workload at the required performance, risk and cost? And how easily can we change that decision later?

This guide provides a practical framework for answering that question.

Start with the workload, not the provider

A neocloud is not a general-purpose upgrade to an existing cloud environment. Its value depends heavily on the workload placed on it.

Before comparing vendors, classify the use case.

Training and fine-tuning

Training workloads normally require high-density accelerator clusters for a finite period. Their key variables include:

The number and generation of accelerators required.
The communication pattern between GPUs.
The size and location of the training dataset.
Checkpoint frequency and size.
The acceptable time to complete the training run.
Whether the workload can be interrupted and resumed.
The cost of a failed or partially completed run.

For these workloads, the cheapest GPU-hour may not produce the cheapest training run. Queue time, interconnect performance, storage throughput and recovery behaviour can change the economics substantially.

Persistent inference

Production inference has a different profile.

The main questions are usually:

Required requests or tokens per second.
P50, P95 and P99 latency.
Traffic variability.
Availability targets.
Model-loading time.
Minimum capacity required during quiet periods.
Scaling speed during demand spikes.
Regional and data-residency requirements.

Inference can run continuously, making poor utilization or overprovisioning much more expensive over time than it would be during a short training project.

Batch processing and HPC

Rendering, simulation, scientific computing and large batch workloads can also be suitable for neocloud infrastructure.

These workloads are often easier to move because they have a clear beginning and end. However, they may depend heavily on specialized filesystems, schedulers, compilers or network configurations.

Burst capacity

A neocloud can supplement an internal cluster or a primary cloud when normal capacity is exhausted.

This model can reduce the need to size private infrastructure for occasional peaks, but it requires the workload to be genuinely portable before the peak occurs. Attempting to create portability during a capacity emergency is rarely successful.

Sovereign or regulated workloads

Some neocloud providers focus on regional or sovereign infrastructure. These services may be relevant when data, operational control or governance must remain within a specific jurisdiction.

In this case, technical performance is only one part of the evaluation. Contractual guarantees, operator access, support location, encryption-key control and applicable law can be equally important.

Create a workload profile

For each candidate workload, produce a one-page profile containing:

Business owner and technical owner.
Workload type.
Current execution environment.
Accelerator requirements.
CPU and memory requirements.
Dataset and checkpoint sizes.
Network characteristics.
Storage characteristics.
Normal and peak capacity.
Expected duration or monthly volume.
Latency and availability objectives.
Data classification.
Geographic restrictions.
Maximum acceptable interruption.
Dependencies on proprietary services.
Target cost metric.
Portability objective.

The target cost metric should reflect the outcome, not the infrastructure unit.

For training, this could be:

Total cost per successful training run.

For inference:

Cost per million tokens at the required latency and availability.

For rendering:

Cost per completed frame.

For simulation:

Cost per completed scenario.

This gives every provider the same problem to solve.

Establish a baseline before requesting quotes

A proposal is difficult to evaluate without knowing how the workload performs today.

Measure the current environment before moving anything.

At minimum, collect:

Time required to obtain capacity.
Environment setup time.
Data-transfer time.
Job execution time.
Average and peak accelerator utilization.
CPU, memory and storage utilization.
Network throughput and latency.
Idle capacity.
Number of failed or repeated jobs.
Engineering hours required to prepare and operate the workload.
Full infrastructure cost.

Avoid comparing the provider’s estimate with an undocumented impression of the current environment.

The baseline does not need to be perfect. It needs to be reproducible.

Define acceptance criteria in advance

Before contacting providers, separate requirements into three categories.

Mandatory requirements

A provider is excluded if it cannot satisfy these conditions.

Examples include:

A specific accelerator type or minimum memory capacity.
Processing within an approved jurisdiction.
Support for a required scheduler.
Customer-managed encryption keys.
A minimum availability commitment.
Checkpoint export in a usable format.
A defined incident-response process.
A contractual right to retrieve all data after termination.

Scored requirements

These requirements influence the ranking but are not automatic exclusions.

Examples include:

Provisioning speed.
Hourly cost.
Reservation flexibility.
Storage performance.
Quality of monitoring.
Support responsiveness.
Available regions.
Sustainability reporting.
Integration with existing tooling.

Disqualifying conditions

Write down the conditions that make a proposal unacceptable, even when its price is attractive.

Examples include:

Unclear data-egress terms.
No documented exit process.
Provider-controlled encryption keys for sensitive workloads.
Unspecified accelerator substitutions.
An SLA that excludes queue or provisioning time.
Take-or-pay commitments that greatly exceed expected utilization.
No ability to reproduce the environment outside the provider.

Defining these conditions before commercial negotiations reduces the risk of changing the evaluation criteria to justify an attractive quote.

Use a weighted provider scorecard

A scorecard forces technical, financial and risk teams to evaluate the same proposal.

A sample weighting could be:

Evaluation area	Suggested weight
Workload performance	25%
Total cost and commercial model	20%
Capacity availability	15%
Security, compliance and sovereignty	15%
Portability and exit readiness	15%
Provider and operational risk	10%

The weights should change according to the use case.

A regulated inference service may place more weight on sovereignty and availability. A temporary training project may prioritize time to capacity and total execution cost.

The scorecard should also include a confidence level. A benchmarked result deserves more confidence than a claim in a presentation.

Evaluate the architecture behind the GPU

A GPU model name does not describe a complete system.

Two providers offering the same accelerator can produce very different results because of network topology, CPU allocation, storage, virtualization and scheduling.

Ask each provider to explain the following.

Accelerator configuration

Exact GPU or accelerator model.
Memory per accelerator.
Number of accelerators per node.
Interconnect inside the node.
Interconnect between nodes.
Whether accelerators are dedicated or shared.
Whether fractional allocation is used.
Whether hardware substitutions are permitted.
How failed hardware is replaced.

Compute environment

Bare metal, virtual machine or container.
CPU and memory allocated per accelerator.
NUMA topology.
Supported operating systems and images.
Driver and CUDA versions.
Upgrade and patching policy.
Support for custom images.
Support for Kubernetes, Slurm or other required schedulers.

Storage

Local ephemeral storage.
Persistent block storage.
Object storage.
Parallel filesystem options.
Read and write throughput.
Metadata performance.
Snapshot and replication capabilities.
Checkpoint recovery time.
Cost of retaining and exporting data.

Operations

Provisioning mechanism.
Infrastructure-as-code support.
API and CLI coverage.
Monitoring and logging.
Alerting.
Access controls.
Audit trails.
Incident communication.
Support escalation.
Capacity-reservation process.

A provider that cannot clearly describe the architecture should not be evaluated solely on its advertised accelerator price.

Run a proof of value, not a synthetic benchmark

Synthetic benchmarks can be useful for identifying obvious hardware problems. They rarely represent the complete economics of a production workload.

A proof of value should use a representative application, dataset and operational process.

Use the same:

Model or application build.
Container image.
Dataset.
Precision.
Batch size.
Checkpoint configuration.
Storage pattern.
Availability target.
Measurement period.

Run the test across the current environment and two or three candidate alternatives.

Phase 1: Provisioning

Measure:

Time from request to usable capacity.
Number of manual steps.
Quota or approval requirements.
Configuration failures.
Reproducibility.

Phase 2: Data preparation

Measure:

Data-ingest time.
Transfer costs.
Storage configuration effort.
Dataset validation.
Time required to make the workload executable.

Phase 3: Execution

Measure:

Total job duration.
Accelerator utilization.
Throughput.
Latency, where relevant.
CPU and storage bottlenecks.
Scaling efficiency across multiple nodes.
Cost while the workload is actively running.

Phase 4: Interruption and recovery

Deliberately test:

Checkpoint creation.
Node failure.
Job restart.
Capacity interruption.
Recovery-point objective.
Recovery-time objective.

A platform that performs well only when nothing fails is not production-ready.

Phase 5: Export and teardown

Measure:

Time to export checkpoints and results.
Egress charges.
Ability to reconstruct the workload elsewhere.
Resource-deletion behaviour.
Residual storage and network costs.
Account-closure process.

The export test is often more informative than the initial deployment.

Calculate the real cost

The total cost of a neocloud workload is not the advertised GPU rate.

A useful model is:

Effective workload cost = compute + attached infrastructure + data movement + unused commitments + operational labour + failures and retries + migration and exit costs

Compute

Include:

Accelerator hours.
CPU and memory.
Minimum node sizes.
Idle time while waiting for data or other jobs.
Capacity retained for availability.
Premiums for specific regions or hardware.

Attached infrastructure

Include:

Persistent storage.
Object storage.
High-performance filesystems.
Snapshots.
Public IP addresses.
Load balancers.
Monitoring.
Logging.
Security services.
Support plans.

Data movement

Include:

Initial dataset upload.
Replication between regions.
Checkpoint transfers.
Result export.
Backup copies.
Egress to another provider or an on-premises environment.

Commitments

Calculate:

Minimum monthly spend.
Reservation period.
Take-or-pay exposure.
Unused reserved capacity.
Renewal conditions.
Early-termination costs.

Operations

Measure the human effort required for:

Environment setup.
Image maintenance.
Security reviews.
Troubleshooting.
Capacity planning.
Cost allocation.
Reporting.
Vendor coordination.

Gartner predicts that, by 2030, organizations that fail to optimize their underlying AI compute environments could pay more than 50% more than those that do.

The lesson is not that one provider category is always cheaper. It is that infrastructure efficiency must be measured continuously.

Compare outcomes rather than hourly rates

For training, compare:

Time to available cluster.
Time to completed model.
Cost per successful run.
Percentage of runs repeated.
Engineering time per run.
Cost of retaining checkpoints.

For inference, compare:

Cost per million tokens or requests.
Throughput at the required latency.
Cost of minimum always-on capacity.
Scaling response time.
Cost of redundancy.
Failure rate.
Availability achieved.

For batch workloads, compare:

Cost per completed batch.
Completion time.
Percentage of capacity left idle.
Queue time.
Retry cost.

A provider that costs 15% more per accelerator-hour may still be more economical if the workload completes 25% faster or requires less operational effort.

The opposite is also true: a low hourly rate may hide poor utilization, slow storage or expensive data movement.

Include private infrastructure in the comparison

The neocloud-versus-hyperscaler comparison is incomplete when private infrastructure is excluded.

IDC reports that worldwide server spending grew by 30.7% year over year in the first quarter of 2026, while unit shipments grew by only 3.3%. IDC attributes much of the difference to continued GPU-server deployment and expects memory and NAND constraints to keep pricing elevated through at least the first half of 2027.

This makes private AI infrastructure expensive to acquire, but it does not automatically make it uneconomical.

Private capacity can be attractive when:

Demand is predictable.
Utilization can remain high.
Data is expensive or difficult to move.
The hardware can support several workloads.
The organization has the skills to operate it.
The depreciation period matches the expected technology lifecycle.
Direct control has security or regulatory value.

Include at least three scenarios in the business case:

External neocloud capacity.
Hyperscaler capacity.
Private or bare-metal capacity.

For many organizations, the best result will be a combination rather than a single winner.

Test portability before signing the contract

Portability should be demonstrated, not declared.

Application portability

Confirm that the workload can be rebuilt from:

Version-controlled code.
Portable container images.
Documented dependencies.
External configuration.
Infrastructure-as-code definitions.
Standard model formats.
Exportable checkpoints.

Data portability

Confirm:

Export formats.
Egress bandwidth.
Egress charges.
Time required to retrieve all data.
Snapshot compatibility.
Backup ownership.
Data-deletion procedures.
Retention after contract termination.

Identity portability

Avoid embedding the provider’s identity model so deeply that every application must be rewritten to move.

Document:

User and service identities.
Role mappings.
Secrets.
Certificates.
API credentials.
Audit requirements.

Operational portability

Export:

Logs.
Metrics.
Dashboards.
Alerting rules.
Runbooks.
Incident histories.
Cost-allocation data.

An application that can technically move but loses all its operational context is only partially portable.

Perform an exit drill

Before committing significant production workloads, recreate the environment somewhere else.

The exit drill should test whether the team can:

Provision an equivalent environment.
Restore a checkpoint.
Reconnect storage.
Recreate network policies.
Restore secrets and identities.
Resume the workload.
Validate the results.
Decommission the original environment.

Record the duration, manual steps and failures.

The purpose is not to move providers immediately. It is to quantify the dependency being created.

Review the commercial exit terms

Technical portability is ineffective when the contract prevents its use.

Review:

Contract duration.
Automatic renewal.
Minimum spend.
Hardware reservation.
Price-adjustment clauses.
Egress pricing.
Support during termination.
Data-retention period.
Data-deletion certification.
Transition assistance.
Rights following provider acquisition or insolvency.
Changes to region or hardware availability.
Subcontractor use.
Applicable law and jurisdiction.

Procurement, security, legal and infrastructure teams should review the exit plan together.

Choose the right operating pattern

A neocloud can be introduced in several ways.

Pattern 1: Specialist training capacity

The provider is used for finite training or fine-tuning projects.

Best for: Projects requiring large clusters for limited periods.

Main advantage: Access to specialized hardware without purchasing it.

Main risk: Data-transfer and environment-setup overhead.

Pattern 2: Private baseline with external burst

Private infrastructure handles predictable demand. A neocloud absorbs peaks or provides access to hardware that is not available internally.

Best for: Organizations with stable baseline utilization and occasional large projects.

Main advantage: Combines infrastructure ownership with external elasticity.

Main risk: Requires workload portability and capacity coordination.

Pattern 3: Multi-provider inference

Production inference runs across two or more providers for resilience, geography or commercial flexibility.

Best for: High-volume services with clear abstraction between the application and infrastructure.

Main advantage: Reduced dependency on a single capacity source.

Main risk: Operational complexity and inconsistent performance.

Pattern 4: Sovereign workload placement

Sensitive workloads are assigned to a regional or sovereign provider while other applications remain in global cloud environments.

Best for: Regulated data and jurisdiction-specific requirements.

Main advantage: Greater control over data location and operations.

Main risk: Smaller service catalogue or more limited capacity.

Pattern 5: Temporary project environment

A neocloud is used for a defined experiment, migration or product-development phase.

Best for: Teams that need capacity quickly but do not yet know their long-term demand.

Main advantage: Limits initial commitment.

Main risk: A temporary environment can quietly become permanent without an exit decision.

The best first workload

The first neocloud workload should be:

Technically representative.
Economically measurable.
Portable.
Important enough to produce useful evidence.
Small enough to limit risk.
Relatively independent from proprietary managed services.
Able to tolerate a controlled test window.

A useful first project is often a training, rendering or batch workload with a known dataset and a clear completion metric.

The first project should not normally be the most critical production service in the organization.

A 30-day neocloud evaluation plan

Week 1: Scope and baseline

Select one representative workload.
Assign business, infrastructure, security and finance owners.
Document technical requirements.
Measure the current environment.
Define the target cost metric.
Write mandatory and disqualifying requirements.

Deliverable: Workload profile and current-state baseline.

Week 2: Provider and architecture review

Send the same requirements to each provider.
Review accelerator, network and storage architecture.
Complete an initial security and sovereignty assessment.
Review pricing and contract structure.
Select two or three providers for testing.

Deliverable: Shortlist and weighted scorecard.

Week 3: Proof of value

Deploy the same workload.
Measure provisioning and data-transfer time.
Run the application.
Test interruption and recovery.
Export the results.
Calculate total workload cost.

Deliverable: Comparable technical and economic results.

Week 4: Decision and operating model

Rank the providers.
Select the intended operating pattern.
Complete legal and commercial review.
Define portability requirements.
Run or schedule the exit drill.
Establish cost and performance monitoring.
Decide whether to expand, repeat or stop.

Deliverable: Adoption decision with an explicit exit plan.

Common mistakes to avoid

Selecting on GPU price alone

An hourly rate does not capture queue time, storage, networking, utilization, support or migration cost.

Benchmarking only the accelerator

The workload runs on a system, not on an isolated chip. Network, storage, CPU and scheduler behaviour matter.

Moving too much data

A workload with a low compute-to-data ratio may be a poor candidate for an external provider, even when compute prices are attractive.

Ignoring unused commitments

Reserved capacity creates value only when it is used. Measure utilization against the committed amount, not merely against allocated resources.

Treating multicloud as portability

Using several providers does not mean workloads can move between them. Multiple non-interoperable environments can increase lock-in rather than reduce it.

Postponing the exit plan

The easiest moment to negotiate export rights and test portability is before the provider hosts critical data.

Assuming every neocloud is equivalent

Providers differ in scale, regions, hardware, software, support, financing and operational maturity. The category name is not a guarantee.

When a neocloud may not be the right choice

A neocloud may provide limited value when:

The workload is small and highly sporadic.
The application depends deeply on proprietary hyperscaler services.
Most of the relevant data already resides in another cloud.
Data movement would dominate the project cost.
Existing infrastructure has sufficient available capacity.
The team needs a highly managed service rather than infrastructure.
The provider cannot meet security or sovereignty requirements.
There is no credible technical or contractual exit path.

The objective is not to introduce another provider. It is to improve the economics, performance or resilience of a specific workload.

From provider selection to continuous workload placement

A neocloud evaluation is not complete when the contract is signed.

Hardware availability, demand, application design and pricing will continue to change. The placement decision should therefore be reviewed regularly.

Track:

Effective cost per workload outcome.
Accelerator utilization.
Idle committed capacity.
Queue and provisioning time.
Failure and retry cost.
Storage and egress growth.
Latency and throughput.
Support incidents.
Portability-test results.
Alternative-provider availability.

The infrastructure that is best today may not remain the best option after the model, accelerator generation or commercial terms change.

The long-term capability to build is not loyalty to a particular provider.

It is the ability to evaluate, place and move workloads using measurable evidence.

Optional tooling for a portable operating model

The evaluation framework above does not depend on Elemento or on any other product. Provider selection, benchmarking, TCO analysis and exit planning should be completed independently.

Tooling becomes relevant after the organization has defined which environments it needs to operate and how much consistency it requires between them.

Electros: when the problem is operational fragmentation

Electros can be evaluated when infrastructure teams need a common operational surface across supported public-cloud and on-premises environments.

It provides a desktop interface and CLI for provisioning, monitoring and accessing virtual machines across AWS, Microsoft Azure, Google Cloud and on-premises infrastructure.

This can be useful in an operating model where neocloud capacity is only one part of a wider infrastructure estate. However, support for any specific provider or workflow should always be validated during the technical evaluation.

Reduce console fragmentation and evaluate a common control experience across your supported cloud and private environments.

Explore Electros

AtomOS: when a private baseline belongs in the model

AtomOS can be evaluated when predictable workloads, sensitive data or stable capacity requirements justify a private or bare-metal infrastructure baseline.

It is a KVM-based, RHEL-compatible hypervisor designed for bare-metal, GPU-enabled and hybrid infrastructure, with GPU passthrough and autonomous clustering without a mandatory central controller.

In a baseline-and-burst model, private capacity can host predictable demand while external providers supply temporary or specialized compute.

Assess whether a KVM-based private infrastructure layer could support the stable portion of your AI and enterprise workloads.

Discover AtomOS

Get an independent view of the architecture

An external review can be useful when several infrastructure options appear technically valid but differ in commercial exposure, portability or operational complexity.

The most productive assessment starts with evidence. Bring:

One representative workload.
Its present performance baseline.
Current or estimated infrastructure costs.
Data-residency requirements.
Existing cloud and virtualization dependencies.
Expected growth.
Contractual constraints.

Elemento offers technical assessments for organizations evaluating public cloud, private infrastructure, hypervisors and cross-provider operating models.

Review your workload profile, target operating model and evaluation plan with the Elemento team before committing to a provider or capacity contract.

Book a Technical Call

The final decision should remain reversible

The purpose of a neocloud strategy is not to replace one default provider with another.

It is to create another credible infrastructure option.

A sound evaluation begins with a workload, measures the complete cost of running it, verifies the underlying architecture, tests failure and recovery, and proves that the application can leave.

This approach may lead to a neocloud, a hyperscaler, private infrastructure or a combination of the three.

The specific answer matters less than the decision process.

When workloads are portable, costs are observable and exits are tested, infrastructure becomes a choice that can be revisited, not a dependency that has to be defended.