Exploring the Effectiveness of Deep-Learning Cellular Segmentation with Cellpose: Application to IMC for Prostate Biopsies
Lauren Sadka
Senior Thesis | 2026
ExploringtheEffectivenessofDeep-LearningCellular SegmentationwithCellpose:ApplicationtoIMCfor ProstateBiopsies
LaurenSadka
BostonUniversityAcademy (researchperformedatMGHVaccine&Immunotherapy Center)
January2026
Background
Accuratecellularsegmentationisanimportantstepinbiomedical imageanalysis,enablingresearcherstoextractmeaningfuldata fromcomplexmicroscopyimages.Cellsegmentationisthe processofidentifyingandoutliningindividualcellswithinan image,andwhendonewell,thecomputercandistinguishbetween individualcells,evenwhentheyaretightlypackedorirregularin shape.Cellsegmentationallowsustoextractquantitativedatafor downstreamanalysis,likespatialrelationshipsandcelltype abundance.Thoughtherearemanyexistingtools,cell segmentationcanstillbedifficultbecauseoftheinherent variabilityofbiologicalcells;thesegmentationsoftwaresmaynot betrainedonthetypesofcellsintheparticulardataset.Inaddition, whensegmentationtoolsaretrainedtobegeneralised,itoften meansthattheydon’tworkwellwithmanyspecificcelltypes. Withnewadvancesintechnology,therearebothmachine-learning, andevenmorerecently,deep-learningbasedsegmentation softwares,buttheyarenotwellcompared.
Traditionalsegmentationapproachesthathavebeendeveloped overthepastseveraldecadesrelyonrule-basedimageprocessing techniquessuchasthresholding,edgedetection,regiongrowing,or watershedalgorithms(Meijering,2012).Thesemethodsattemptto isolateindividualcellsbyidentifyingdifferencesinpixelintensity, shape,ortexture.Forexample,thresholdingsegmentsanimageby separatingdarkerandlighterregionsassumingthatcellsand backgrounddiffersignificantlyinbrightness.Watershedmethods, however,inspiredbytopography/morphology,treattheimageasa surfaceandidentify“basins”thatlikelycorrespondtoindividual cells.
Whilesuchapproachescanworkwellforsimplehigh-contrast imagesthathavehighSignal-to-NoiseRatios(SNRs),forexample,
isolatedcellsonaclearbackground,theystrugglewithreal-world biomedicaldataliketissuesamples.Inprostatetumors,for example,cellsareoftendenselypacked,irregularlyshaped,and overlapping.Numerouscelltypes,eachwiththeirdistinctshapes comprisethebiologicalstructuressuchasglands,veins,and stromaltissuetogetherwithvariationsinlabelingofnucleiwith DNAintercalatorsandantibodylabelingofproteinsofinterestcan furthercomplicatethesegmentation.Rule-basedalgorithmslack theabilitytoadapttothesecomplexitiesandoftenrequire elaboratetuningforeachnewdatasetorimagingmodality.
Theintegrationofmachinelearningintoimagesegmentationinthe 2010sledtonewcellularsegmentationmethods.Unlikethe traditionalmethods,machinelearningdoesn’trelyonpredefined rules.Instead,itlearnspatternsdirectlyfromannotatedtraining data,andusesamethodcalledpixelclassification.Insupervised learningpipelines,aresearchermustmanuallylabelasubsetof cellsinsampleimages.Thealgorithmthenlearnstoclassifypixels basedontheseexamples,usingfeatureslikeintensity,shape,or texture.Oneofthemostwell-knowntoolsusingthistechniqueis Ilastik,whichusesauser-friendlygraphicalinterface(Bergetal., 2019).Inthescientificliterature,thesemodelsarecharacterizedas relativelyfasttotrainandeffectiveinmoderatelycomplex datasets,thoughtheystillrelyonmanuallyannotatedfeaturesand limitedlearningcapacity,whichcanlimitthegeneralisationof thesemodelstonewtypesoftissue.
However,here,thepurposeofthecellularsegmentationisto accuratelydeterminethebordersforeachindividualcell,atask thatgoesbeyondpixelclassification(forexample,pixelsfor nuclei,cytosol,andbackground).Whilepixelclassification adetermineswhateachpixelrepresents,accuratesegmentation mustreconstructfullcellboundaries,whichisespecially
challengingincomplextumortissuewherenucleiaretightly clusteredandirregularlyshaped.Thus,Ilastikmustbeusedin conjunctionwithanotheralgorithmthatwillcreatethecelloutlines usingtheinformationfromIlastikaboutwhichpixelsrepresent nuclei,cytosol,andbackground.Inourcase,CellProfilerwas selectedforthistask.CellProfilerisanopen-source,modular imageanalysissoftwaredesignedtoextractquantitative measurementsfrombiologicalimagesbyapplyingasequenceof user-definedprocessingsteps,referredtoasa“pipeline”(Stirling etal.,2021).Inthecontextofcellularsegmentation,CellProfiler usesthepixel-levelclassificationsproducedbyIlastikasinputand appliesobject-identificationalgorithmstogroupclassifiedpixels intodiscretecellularobjects.Thesestepsincludeidentifying primaryobjects(suchasnuclei),expandingorassociating secondaryobjects(suchasthecytoplasm),andrefiningobject boundariesthroughsize,shape,andintensityconstraints.
UnlikeIlastik,whichperformspixelclassification,CellProfileris usedforconvertingtheclassificationsintobiologicallymeaningful celloutlinesandforgeneratingdownstreamquantitative measurementssuchasobject(cell)count,area,shape,andpixel intensitiesperimmunomarker.Thus,CellProfilerisanecessary stepinthesegmentationpipeline,asitisneededtobridgethegap betweenpixelclassificationandsingle-cell-levelanalysis. However,constructinganeffectiveCellProfilerpipelinerequires carefulparameterselectionanditerativetuning,andperformance canvarydependingonimagequalityandtissuecomplexity (Stirlingetal.,2021).
Thisstudyaimsforasimplersolutiontocellsegmentation:one thatperformsallthetasks(eliminatingtheneedforapipelineof twosoftwares),onethatdoesn’trequireanexcessiveamountof human-in-the-looptrainingormanualannotationsandthusquicker
totrain.Newuser-friendly,novelsegmentationmethodsthat integratedeeplearningcanhelp.
Deeplearningapproachestheimagesegmentationproblem differently.Itenablesmodelstolearnrepresentationsdirectlyfrom therawdatainsteadofrelyingonmanuallyannotatedfeatures.In imagesegmentationtasks,thistypicallyinvolvesusing convolutionalneuralnetworks(CNNs)toidentifypatternsand structuresthatcorrespondtothecellboundaries.CNNsworkby applyingfilterslikeedgedetectorsortextureidentifiersthatsearch theimagetopickupmeaningfulpatterns.AccordingtoMoenetal. (2019),deeplearningmodelscanautomaticallyinferhierarchical featuresrelevantforsegmentation,makingthemhighlyadaptable acrossdifferentimagetypesandmodalities.Thesemodelsare trainedend-to-end,meaningthattheentirenetwork,frominput imagetooutputmask,isoptimizedtominimizesegmentation errors,streamliningthelearningprocess.Asnotedin Image Segmentation Using Deep Learning: A Survey,thestructureof thesemodelsisparticularlyeffectiveinbiomedicalimage segmentation,wherefeaturessuchasshape,texture,andintensity distributioncanvaryalot(Moenetal.,2019).Thisway, segmentationalgorithmsusingdeeplearninghavethepotentialto createasingularmodelthatgeneralizeswellwithminimalhuman input(Moenetal,2019).
Arecentstudy, Comparative Study of Cell Nuclei Segmentation
Based on Computational and Handcrafted Features Using Machine Learning Algorithms (2025),comparedseveral machine-learningapproachesforcellnucleisegmentationusing bothmanuallyannotatedfeaturesandCNN-derivedfeatures, acrosstwoprostate cancerhistopathologydatasets(Radboudand MoNuSeg).Thestudycomparedapproachesbasedonthe followingmetrics:accuracy,theDicecoefficient,andtheJaccard
index.Accuracyreflectstheproportionofpixelscorrectly classifiedaseither“cell”or“notcell”(background),butitcanbe misleadinginimageswherethereisalargerpercentageof backgroundpixels.However,accuracycanbemisleadingin imageswherethereisalargerpercentageofbackgroundpixels. Therefore,theDicecoefficientandJaccardindexaremoreuseful metrics,astheybothquantifyhowmuchthepredictedcellmask overlapswiththeground-truth(Ilastik/CellProfiler)mask.The Dicecoefficientemphasizestheoverlapandsimilarity,andthe Jaccardindexprovidesastricterpenaltyformismatchedregions. ThestudyshowedthatusingCNN basedfeaturesachievedthe strongestperformance,reaching96.90%accuracy,aDice coefficientof74.24,andaJaccardindexof55.61.This outperformedRandomForest,SVM,andK meansclustering,all ofwhichproducedlowersegmentationquality(Sumonetal., 2025).Dicecoefficientemphasizestheoverlapsimilarityandthe Jaccardindexprovidesastricterpenaltyformismatchedregions.
LogisticRegressionusingmanuallyannotatedfeaturesalso performedwellbutremainedslightlyinferiorto CNN feature basedmodels,achieving96.10%accuracy,Dice 68.32,andJaccard53.81(Sumonetal.,2025).Theauthors concludethatCNN-derivedrepresentations(deeplearning) substantiallyenhancesegmentationaccuracy,whiletraditional machine-learningmodelsrelyingonmanualannotationsstruggle morewithcomplexnuclearmorphologyandstainvariabilityin prostatecancertissue(Sumonetal,2025).
Introduction
ThispaperexplorestheeffectivenessofCellpose,ageneralist, deeplearning-basedsegmentationtool,inanalyzingprostatetumor imagesandcomparesittotraditionalmachinelearningpipelines usedinthelab.(Stringer,C.etal.,2021).Cellposeusesaneural
networktopredicttheflowofpixelsfromthecenterofeachcellto itsboundary,whiletraditionalmethodsrelyonedgedetection instead.Cellposeworksinthreesteps:Flowprediction,inwhich themodelpredictshowpixelsmovefromthecentertothe boundary;maskgeneration,whereCellposeconstructsmasksfor eachcellusingtheflowdata;andoptimization,whichrefinesthe maskwithpost-processingtechniques.TheutilityofCellposelies initspretrainedmodels,aswellitssupportforcustom training—whichultimatelyenablesuserstofine-tunemodelsfor domain-specificdatasets.Theabilitytoadjustmultipleparameters simultaneouslyallowsforgreaterfine-tuningthanispossiblewith manyothersegmentationtools(Pachitariu,M.&Stringer,C., 2022).
Assegmentationqualitydirectlyimpactsthereliabilityof cell-basedquantificationandspatialanalyses,evaluatingthe performanceandadaptabilityofdifferentCellposetraining strategiesiscriticaltoadvancingsingle-cellresolutionstudiesin complextissues.
Unlikedeeplearning,Ilastikdoesnotbenefitfromlargedense annotations.Itworksbestwhenlabelsareaddedinteractivelyto correctmistakes.Poorlyplacedannotationscanmisleadthe classifier,sothemanualannotationsmustbeprecise(Bergetal., 2019).
Instudyingprostatecancertissuesamples,stainedlabeledwith49 antibodiesidentifyingdistinctmolecularmarkersoftissue structuresandcellphenotypes,itischallengingtoextractaccurate, cell-levelinformationfromdenselypackedandoftenirregular tumorstructures.Ifapatientissuspectedofhavingprostatecancer basedonaPSAtestorphysicalexam,diagnosisrequiresaneedle biopsytoassessmorphologicalchangesinthetissue.
HistopathologistsusetheH&E(HematoxylinandEosin)staining
nucleiblue-purpleandcytoplasmpink,tovisualizetissue architectureforhallmarksthatdeterminecancerpresenceand severity.WhileH&Estainingisessentialforidentifyingstructural changesintissue,itoffersnomolecularorprotein-level informationaboutthecells.ImagingMassCytometryaddresses thelimitationsofH&Estainingbyenablingmultiplexeddetection ofuptomultiplemarkersinasingletissuesection(Giesenetal, 2014).IMCusesmetal-taggedantibodiesandalaserablation systemtovaporizetissuepixel-by-pixel,witheachplumeanalyzed byaCyTOFmassspectrometertoquantifymultipleprotein markerssimultaneouslyatsubcellular(1μm2/pixel)resolution.This multiplexed,quantitativereadoutovercomeskeylimitationsof H&Estainingbyprovidinginformationaboutcelltypes,states, andinteractionsthatH&Edoesn’tdistinguish.Thisallows researcherstovisualizenotonlythestructurebutalsothe molecularphenotypeofindividualcells.Thisstudyfocusedonthe DNAchannelfromtheIMCdatatoguidesegmentation,asevery cellcontainsDNAregardlessofitstypeorfunction.
TotailorCellposetoourdataset,thepretrainedmodelswerefirst exploredtodeterminewhichonewouldprovidethemosteffective foundationforcustomtraining.Thisstudyfocusesonthethree mainpretrainedmodels:Cyto3(generalist)model,trainedto segmentwholecells(inimagesthatmayormaynothaveanuclear channel);Nucleimodel,trainedtosegmentcellnucleiin single-channelimages;and(New!2025)SAM (Segment-Anything-Model):FoundationalSAMmodelintegrated intotheframeworkofCellposeandfine-tunedwithcellularimages (Pachitariu,M.,2025).
Methods
TheaimofthecustomCellposesegmentationmodelisto accuratelyandefficientlysegmentadatasetofaround200IMC stainedmicroscopyimagesofprostatetumorswithapproximately 49immunochemicalmarkers.Thesemarkersincludediridiumfor DNAlabelling,andEosinforcytosollabelling.Thesegmentation wasperformedsolelyontheDNAchanneloftheimages.
Cellposeoffersmultipleparameterselection,whichenablesthe usertothresholdobjectsthatcausenoiseorinsignificant expressionsintheimage.Theabilitytoadjustmultipleparameters simultaneouslyallowsforgreaterfine-tuningthanispossiblewith manyothersegmentationtools.Themainparametersareexplained inTable1:
Table 1: Table providing information about Cellpose’s main parameters
Parameter Function Range
Cellprob threshold Controllinghowconfidentthe modelmustbetoclassifya regionasacell. [-6,6]
Flow threshold Edgedetection:controllinghow aggressivelyCellposeseparates cellsfromthebackgroundand eachother. [0,3]
Parameter Function Range
Average Diameter Averagediameter(pixels)of eachcell/mask.
Cellposeoutputsthreemainelements:aprobabilitymask,which indicatestheconfidencethateachpixelbelongstoacell;anROI count,whichreflectsthenumberofsegmentedcells;anda segmentationmask(savedasa.npyfile),whichassignseachpixel toaspecificcellregionbasedonthefinalsegmentation.The probabilitymaskisgrayscale,representingconfidenceperpixel, whilethesegmentationmaskislabeledwithdistinctintegervalues foreachindividualcell.
Developingastablesegmentationworkflowrequiredextensive iterativeparameterrefinement.IMcimagesarestoredas multi-channelTIFFstacks,andpreprocessingwasneededto correctlyextractandreformattheDNAchannelfor two-dimensionalsegmentation.BasicPythonknowledgewas requiredtowritescriptsthatmanipulatedtheimagestacksto isolatethedesiredchannels,andlater,toextractmetricsfrom Cellposeoutputfiles.Multipleroundsoftroubleshootingwere requiredtoresolveissueslikefindingmoreaccurateparameters, runtimeerrors,andeditingsegmentation.Carefulvisualchecking ofthemaskswasessentialthroughouttheprocesstoensurethat themasksaccuratelyrepresentedthecells.
Firstly,threeofCellpose’spretrainedsegmentationmodelswere selectedtobetestedforuseonourdataset:Cyto3,Nuclei,and SAM.
Figure 1: Close-up of segmentation by various pretrained models. Some relevant areas are circled in red.
Figure1presentsaclose-upofaparticularlychallengingareaof oneoftheimages,segmentedbythe3differentpretrainedmodels withthedefaultsetofparameters,andacustomsetofparameters. Withthedefaultparameters(flowthreshold=0.4andcellprob threshold=0),thepretrainedmodelstendedtomisscells,sothe parameterswerefixedto:averagediameter=9,flowthreshold= 0.7,andcellprobthreshold=-1).
Ultimately,noneofthesemodelswereabletoaccuratelysegment imagesinthedatasetdespiteparametertuning.TheCyto3model andNucleimodelconsistentlymissedcells,andintryingtorelax theparameterstocaptureallofthecells,manybackgroundand noisepixelswereincorrectlyidentifiedascells.TheSAMmodel wasabletoidentifymoreofthecells,butdrewveryinaccurate boundaries,evenwithparametertuning.Inaddition,theSAM modelcantakeupto15minutesperimagewhenrunonaCentral ProcessingUnit(CPU),whichisasignificantdrawbacktothe modelifusingaGraphicsProcessingUnit(GPU)isnotpossible. Inconclusion,thetestingoftheCellpose’spretrainedmodels
showsthat,evenwithparameterrefinement,theCyto3andNuclei modelsareinsufficientforsegmentingthedatasetandtheSAM modelisimpracticalduetoitsruntimeonCPU.
Thus,acustomizedmodelmustbemadeinordertoachieve qualitysegmentationthatcomparestoresultsfromthecurrent Ilastik/CellProfilerpipeline.Inordertocreateacustommodelwith Cellpose,abasemodel(oneofCellpose’spretrainedmodels)must beselected,andthemodelwillbetrainedbasedonanydesired numberofannotatedimages.
TochoosewhichofCellpose’spretrainedmodelstouseasthebase model,threecustommodelsweretrainedwiththesame5 manuallyannotatedimages,buteachwithadifferentbasemodel (onewithCyto3,onewithNuclei,andonewithSAM).After concludingthatthethreepretrainedmodelsperformedvery similarly,thedecisionwasmadetocontinuewiththeNucleimodel becausetheNucleichanneloftheimagesisbeingsegmented.
Withtheaimoflearningabouttheefficiencyandfeasibilityof usingCellpose,thecustommodelswerecreatedusingcropsof wholeimagesinthedataset.57crops,chosentobegenerally representativeoftheentiredataset,weremanuallyannotated.With theaimofevaluatingtheevolutionofCellposesegmentation modelsasthetrainingdatasetisincreased,acustomizedNuclei modelwastrainedonaset19(SetA)manuallyannotatedcrops (Model1).Thismodelwasthenusedtoaidthemanualannotation ofasecondsetof19crops(SetB),andasecondNucleimodel (Model2)wastrainedontheunionofthesetsof19crops(SetA andSetB).Thisprocesswasrepeatedforthethirdsetof19crops (SetC)totrainathird,customizedNucleimodel(Model3).
Theimprovementandevolutionofthethreecustommodelswas assessedbycomparingthecellcountswithIlastik/CellProfiler’s
cellcountsin38imagesfromthedataset,includingbothimages partiallyrepresentedinthetrainingcropsandadditionalimages notdirectlyusedforannotation.
Results
Toassesstheevolutionofsegmentationperformanceacrossthe threecustomCellposemodels,wecomparedthenumberof segmentedregionsofinterest(ROIs),producedbyeachmodelon thesamesetof38testimages.These38testimagesincludedsome imagesthatwereusedinthetrainingdataset,andseveralimages fromthelargerdatasetthatthemodelhadneverseenbefore.As eachmodelwastrainedonprogressivelymorediverseand extensivetrainingdata,atrendofincreasingsegmentation coverage,andbettersegmentationaccuracywasexpected.The primarymetricassessedwasthecellcount(termedsegmented ROIsinCellpose)perimage.Figures2and3illustratethese trends.
Figure 2: Number of segmented ROIs across 38 test images for Models 1–3, demonstrating progressive improvement in segmentation coverage.
InFigure2,theIlastikmodel’scellcountswereusedasa benchmarktoanalysetheperformanceofthethree models:Model1(trainedon19images),Model2(trainedon 38images)andModel3(trainedon57images).Thecell countperimagepermodelisgraphedwithadiagonal45॰ linethatvisuallyrepresentstheIlastikcellcountsasthe benchmark,andthetwo10%boundsareoutlinedinthe orangedottedlines.Everytestedimagecontainsthree points,abluedotrepresentingthenumberofcellsfound byModel1,anorangeboxrepresentingthenumberofcells foundbyModel2,andagreencrossrepresentingthe numberofcellsfoundbyModel3.Thesethreepointsare alignedverticallyforeachimage.Thethreemodelswere runwithidenticalparameterson37randomlychosen wholeimagesfromthedataset,andthecellcountswere compared.Todeterminethegeneralperformanceofthese models,theaveragepercenterrorforalloftheimagesand forsetsofimagesbasedonthenumberofcellswere calculatedforeachmodel,andpresentedinFigure3.
Figure 3: Average absolute percent difference in ROI Counts for Model 1, 2, and 3, vs Ilastik, calculated for sets of images with ROI count: below 9k, below 6k, above 9k, and below 5k. A total percent difference for all images was also calculated per model.
AccordingtoFigure3,theaverageabsoluterelativedistanceto Ilastikis7.80%,10%,and8.32%forModels,1,2,and3 respectively.Theaverageabsoluterelativedifferencestendtobe generallylowerforlargeROIcounts(above9k)acrossallmodels, butthelowestformodel1.ForallotherROIcountranges,model 1andmodel3seemtobecomparable,model2exhibits significantlyhigherrelativeabsolutedifferences.
Discussion
Theresultssuggestthat,forthestudiedsampleofimages,Model1 trainedbyonly19imagesseemstobesufficient—thereisno apparentneedtogoontoModels2and3,eachtrainedbyan additional19crops.Overall,theimplicationisthatthe deep-learningsoftwareCellposecanachievesimilarresultsto machine-learningsoftwareIlastikwithonlyonethirdofthe trainingdata.
Inaddition,forsomepurposes,Cellposewaseasiertousethanthe Ilastik/CellProfilerpipeline,aspertheelementsinTable2:
Table 2: Usability of Cellpose, Ilastik, and CellProfiler by availability of features Tool
WhenCellpose,Ilastik,andCellProfilerarecomparedforeaseof useability,Cellposeendsupbeingtheeasiesttousewhenjust lookingatthebasicfeaturesitoffers.Thesoftwarecanreach segmentationresultsimmediately(withnomanualtraining)using anyofthe20+pretrainedmodelsnowavailable.Ifthe segmentationisalmostsatisfactorybutnotquitethereyet,there
aremultipleparametersthatcanbeadjustedtorefineboththe segmentationandtheoriginalimage;Cellposeincludesimage restorationparameterssuchas‘filter’,‘denoise’,‘deblur’, ‘upsample’,andaunique‘one-click’parameterthataimstorefine theentireimageinonego.CreatingacustommodelinCellpose requiresmanualsegmentation,butitessentiallyinvolvesrefininga selectedpretrainedmodelwiththemanualannotations. Additionally,allofthesefeaturesareavailableinCellpose’s GraphicalUserInterface(GUI),whichenablesuserswhoarenot equippedwiththerequiredcodingskillsforitsCommandLine Interface(CLI)alternative.
AsillustratedinTable2,IlastikalsousesaGUI,whichisan advantageforbeginners.However,Ilastikdoesnotprovideany pretrainedmodels,whichmeansthatthecustomsegmentation modelismadeupentirelyoftheuser’smanualannotations.This meansthattraininganIlastikmodelrequiresmoremanuallabour andcouldbewhyCellposewasabletoachievesimilarresultswith lesstraining.Additionally,sinceIlastikemployspixel classification,theuserhastomarkareasforeachlabel,whichcan takelongerorbemorecomplicatedthanoutliningthefullcellas Cellposerequires,butresultsinIlastikbeingabletoidentifywhere partsofthecellsare,suchasthenucleusorcysoplasm—Cellpose cannotprovidesimilarlevelofdetailabouttheactualcellbesides whichpixelsareattributedtoit.However,Ilastikdoesnotoutput thefullcelloutlines,rather,probabilitymaps–multichannelfloat imagesshowingclasslikelihood–,CSVsthatcontainsdata includingtrackID,time,center,area,etc.,foreachobject,and simplesegmentations–single-channellabelimageswherepixel valuesrepresentclassIDs(Bergetal.,2019).Thus,asecond algorithmisthenneededtoperformthesegmentation,whichis additionallabor.CellProfiler,whichalsoincludesauser-friendly GUI,ishelpfultoapplyimage-processingparameterslike
illuminationcorrection,smoothingandfiltering,thresholding choices,objectidentificationrules(eg.declumping,size constraints).Italsooffersobject(cell)segmentationparameters likewatershed-basedsegmentation,andspecificmeasurement moduleparameters,thatallowuserstochoosewhichobjectsto measure,whichmeasurementcategoriestocompute,andwhether tomeasureper image,per object,orexperiment levelfeatures (Stirlingetal.,2021).
CellProfileralsoreportsmoreinformationaboutcells,including cellmorphology(e.g.area,perimeter,shapedescriptors,and3D volumeforvolumetricdata),intensitymeasurementslikemean, median,andintegratedintensity,intensitydistributionacrossthe object,andChannel specificfluorescencelevels(essentialfor assayslikereporterexpressionorproteinlocalization),andmore informationabouttexturefeatures,colocalizationand multi-channelrelationships,celltracking,andsummarystatistics perimage,includingcellcounts.(Stirlingetal.,2021).Although Cellposedoesn’treportthesestatisticsanddatadirectly,its segmentationresultscanbeexportedasa.npyfile,whichcanbe processedwithpython,andmuchofthisdatacanbeextracted. However,thisdoesrequireadditionalPython/MATLAB knowledge.
AnaspectofIlastik’s(andCellProfiler’s)GUItonotethatis particularlyusefulistheabilitytorapidscanthroughimages directlyinsidetheviewer(e.g.xylayersofaz-stackimageor betweenmultipleimages),andoptionthatCellposedoesnot currentlyprovide.Thisfeatureisusefulwhenworkingwith multiplexeddata,becausemultipledifferentchannelsorslicescan beselectedandviewedsimultaneously,andthesoftwareupdates thedisplayinstantly.An“xylayer”referstoatwo-dimensional imageslice,whilea“z-stack”traditionallyreferstosequential
opticalslicescapturedatdifferentdepths.InIMC,however,the so-calledz-stackcorrespondsnottophysicaldepthbuttomultiple molecularmarkerchannelsacquiredatthesamespatiallocation. WhileCellposetechnicallysupportsZ-stacks,itdoesnotofferthe samelevelofinteractivecontrol.Itisnotpossibletoinstruct Cellposetosegmentbasedonaparticularselectionoflayers,nor tooverlayortogglechannelsthewaytheycanbeinIlastikand CellProfiler.Theonlyworkaroundistomanuallyextractthe desiredlayersfromyourTIFF,assemblethemintoa2D composite,andthenfeedthatintotheCellpose2Dviewer. However,whenworkingwithIMCdata,wheretheZ-stackis reallyjustacollectionofdifferentimmunomarkerchannelsrather thanaphysicaldepthstack,thisquicklybecomestediousandcan requiremorepythonknowledge.
AfterassemblingacustomTIFFwiththedesiredlayersand channels,Cellposeofferslimitedoptionsforvisualizing multi-channeldata.TheCytomodelrestrictsdisplaytotwocolors: oneforcytoplasmandonefornuclei.Thislimitationmakesit difficulttorepresentadditionalmarkersinameaningfulway.By contrast,Ilastik,andCellProfilerprovideafullmulti-channel imagenatively,allowinguserstotoggle,hide,orblendchannels directlyintheGUI.
Conclusion
Inconclusion,IlastikandCellProfiler’sGUIsarebettersuitedfor multiplexedimagingworkflows(likeIMC),sincetheyallowdirect interactionwiththecompletedatasetratherthanrequiring extensivepreprocessing.
Thisstudydemonstratesthatdeep-learningbasedsegmentation, specificallythroughcustom-trainedCellposemodels,canserveas anefficientandeffectivealternativetothecurrent
machine-learningIlastik&CellProfilerpipelineforsegmenting IMCprostatetumorbiopsies.ThissupportstheresultsfromSumon etal.’spaper.Despitethecomplexityofthetissueandthe challengesinherenttodenselypacked,heterogenoustumor environments,CellposeachievedcellcountscomparabletoIlastik whilerequiringsubstantiallylessmanuallyannotatedtrainingdata. Notably,thefirstcustommodel,trainedon19crops,performed comparablywiththeIlastik/CellProfilerpipeline,equallywiththe Cellposecustommodeltrainedon57crops,andimportantly,better thantheCellposecustommodeltrainedon38crops.Thissuggests thatCellposecangeneralizewellwithmanualhuman-in-the-loop inputwhenthetrainingsetisthoughtfullycurated.
Beyondquantitativeperformance,Cellposeofferedpractical advantagesinusability,flexibility,andsimplifiedtheworkflow.Its integratedGUI,multipleparametercontrols,andsupportforboth pretrainedandcustommodelsstreamlinethesegmentationprocess andreducetheneedformultiplesoftwares.WhileIlastikand CellProfilercanprovemoreinsomeareassuchasmulti-channel visualisationanddetailedmorphologicalmeasurements,this paper’sresultshighlightthepotentialofdeep-learning segmentationtosimplifyandacceleratesingle-cellanalysisin multiplexedimagingstudies.
However,theresultsposeseveralquestionsandcanpointtowards nextstepsforfutureexperiments.Thecomparativelypoor performanceofModel2(trainedon38crops:SetA+B), consistentlyproducinghigherpercenterrorsacrossnearlyall ROI-countranges,isunexpected.Itremainsunclearwhetherthis dropinperformancereflectsissueswiththeactualcropsinSetB, suchasunrepresentedtissueregions,poorimagequality(dueto staining),orinconsistentmanualannotations.Reversingthe trainingorderorcreatingacustommodelbasedonsetBalone couldaddressthesequestions.
Anotherdirectionistomovebeyondnuclearsegmentationand exploreCellpose’sCyto3model,whichincludescytoplasmic markers.Integratingthiscellularinformationmayimprove boundaryaccuracy,especiallyintumormicroenvironmentswhere nucleialonewouldprovidelimitedstructuralinformation.
Similarly,evaluatingCellpose’sSAMmodelcouldbebeneficial. AlthoughtheSAMwasslowerwhenrunonCPU,itsaccuracywas insomecasessuperiortotheotherpretrainedmodels.Running Cellpose-SAMonaGPU-equippedsystemmaydramatically changeitsfeasibilityandperhapseventhesegmentationquality.
Cellpose-SAMholdspromiseasageneralistsegmentation foundationmodel,andcouldpotentiallyoutperformtheCellpose Nucleicustommodelswithouttheneedformuchcustomization (Pachitariu,M.etal,2025).
Thisstudyreliedprimarilyoncellcountsasthemetricfor comparingsegmentationperformance.Whilecellcountsare useful,theydonotcapturepixel-levelaccuracyorboundary accuracy.Amorerigorousevaluationcouldinvolvecalculating pixeloverlapmetricspercell,includingDicecoefficientsandthe Jaccardindex.Thesemetricswouldquantifynotonlyhowmany cellsaredetected,butalsohowwelleachcellissegmented comparedtoIlastik/CellProfiler’ssegmentation.
Overall,thisstudyhighlightsboththepromiseandlimitationsof deep-learningbasedcellularsegmentationincomplextumor microenvironments.WhilecustomizedCellposemodelsachieved cellscountscomparabletotheIlastik/CellProfilerpipelinewitha fewernumberofmanuallyannotatedtrainingimages,equivalence incellcountdoesnotnecessarilyimplyequivalencein segmentationaccuracyorboundaryprecision.Ilastikand CellProfileroffergreatertransparencyandmoredetailed morphologicalmeasurements,butCellposecansimplifythe
workflowbyeliminatingtheburdenofusingtwosoftwaresand reducingthenumberofmanuallyannotatedimagesneededfor training.However,Cellposemightrequiredifferentandpotentially morepriorknowledgetobeabletouseCellposequicklyand efficiently.AsimagingtechnologieslikeIMCcontinuetoimprove incomplexityandscale,segmentationapproacheswillneedto balanceefficiency,interpretability,andbiologicalreliability.This researchrepresentsonesteptowardunderstandinghow deep-learningtoolscanbethoughtfullyintegratedintobiomedical researchwithoutlosingcriticalhumanoversight.
Bibliography
Berg,S.,Kutra,D.,Kroeger,T.,etal.(2019).ilastik: interactivemachinelearningfor(bio)imageanalysis. NatureMethods,16,1226–1232.(ilastik:interactive machinelearningfor(bio)imageanalysis|NatureMethods)
E.Meijering,“Cellsegmentation:50yearsdowntheroad,” IEEE Signal Processing Magazine,vol.29,no.5,pp. 140–145,Sep.2012.(CellSegmentation:50YearsDown theRoad)
GiesenC,WangHA,SchapiroD,ZivanovicN,JacobsA, HattendorfB,SchüfflerPJ,GrolimundD,BuhmannJM, BrandtS,VargaZ,WildPJ,GüntherD,BodenmillerB. Highlymultiplexedimagingoftumortissueswith subcellularresolutionbymasscytometry.NatMethods. 2014Apr;11(4):417-22.doi:10.1038/nmeth.2869.Epub 2014Mar2.PMID:24584193.
Moen,E.,Bannon,D.,Kudo,T.,Graf,W.,Covert,M.,& VanValen,D.(2019).Deeplearningforcellularimage analysis.NatureMethods,16(12),1233-1246.(Deep learningforcellularimageanalysis-PubMed)
Pachitariu,M.,&Stringer,C.(2022).Cellpose2.0:howto trainyourownmodel.NatureMethods,1-8.(Cellpose2.0: howtotrainyourownmodel|NatureMethods)
Pachitariu,M.,Rariden,M.,&Stringer,C.(2025).
Cellpose-SAM:superhumangeneralizationforcellular segmentation.bioRxiv.(Cellpose-SAM:superhuman generalizationforcellularsegmentation|bioRxiv)
Stirling,D.R.,Swain-Bowden,M.J.,Lucas,A.M., Carpenter,A.E.,Cimini,B.A.,&Goodman,A.(2021). CellProfiler4:improvementsinspeed,utilityandusability. BMCBioinformatics,22(1),433.(CiteUs|CellProfiler)
Stringer,C.,&Pachitariu,M.(2025).Cellpose3:one-click imagerestorationforimprovedsegmentation.Nature Methods.
(https://www.biorxiv.org/content/10.1101/2024.02.10.5797 80v1)
Stringer,C.,Wang,T.,Michaelos,M.,&Pachitariu,M. (2021).Cellpose:ageneralistalgorithmforcellular segmentation.NatureMethods,18(1),100-106.
(https://www.nature.com/articles/s41592-020-01018-x)