Poster Session A   |   11:45am Expo - Hall A & C   |   Poster ID #103

Irregularities of the nuclear lamina are a marker of cancer progression

Academic Research
Prevention, Early Detection, Implementation, and Dissemination
FDA Status:
Not Applicable
CPRIT Grant:
Cancer Site(s):
Head and Neck, Melanoma of the skin, Breast, Thyroid
Tanmay P Lele
Texas A&M Engineering Experiment Station
Ting-Ching Wang
Texas A&M Engineering Experiment Station
Christina R Dubell
Texas A&M Engineering Experiment Station
Saptarshi Chakraborty
University of Buffalo


The irregular nuclear shape typically found in cancer cells is widely used by pathologists as a marker of cancer presence and severity. Nuclear shape irregularity is determined qualitatively by examining hematoxylin and eosin (H&E) stained tissues. As hematoxylin directly stains chromatin, it does not delineate the nuclear boundary, which is essential for capturing many nuclear surface irregularities such as folds and wrinkles. Furthermore, the qualitative approach to describing nuclear shape leads to inter- and intra-observer variability. There have been efforts to quantify features of nuclear irregularity from H&E stained tissues, but our approach of quantifying nuclear shape in patient tissue samples immunostained for nuclear lamins overcomes both the limitation of qualitative nuclear assessments and the limitation of the H&E staining methods currently used.


Tissue microarrays and individual patient samples from various cancers were immunostained for Lamin B1, as a nuclear shape marker, and pan-cytokeratin, as an epithelial cell marker, from which carcinomas arise. High resolution images were taken with a confocal fluorescence microscope. The nuclei were segmented using the deep learning program Cellpose, and a custom MATLAB program was used to eliminate nuclei from non-epithelial cells and nuclei that were out of plane. Typically, over a thousand nuclei were included in the analysis per tumor grade. Based on the segmented bulk nuclear masks, the precise nuclear contour was delineated at sub-pixel resolution by tracing the intensity maximum on each normal line along the nuclear periphery. Elliptical Fourier Coefficient (EFC) ratio was calculated to quantify nuclear irregularity by approximating the nuclear contours with a series of elliptic harmonics. Other nuclear shape parameters were also quantified based on precise nuclear segmentation, including nuclear area and aspect ratio. Posterior probability was computed from a linear discriminant analysis of geometric quantifications for discrimination between normal tissue and tumor grades.


Nuclear-peripheral marker staining of lamin B1, coupled with our new computational method, showed a more detailed morphology for better detection of nuclear irregularities compared to DNA staining. The number of harmonic ellipses used in the EFC ratio calculation was optimized based on Fréchet distance, which quantifies the accuracy of the Fourier analysis to reconstruct the nuclear periphery. We applied this computational tool to analyze images of human carcinoma and cancer adjacent tissue from various sites, including breast, head and neck, colon, thyroid, ovary, and fallopian tubes. Consistent with previous literature, cancer nuclear sizes were significantly larger than adjacent tissue in all the cancer types. EFC ratio quantification exhibited separations between adjacent tissue and other tumor grades, suggesting the feasibility of statistically comparing the degree of drop-like nuclear irregularity. An EFC ratio value of approximately 4 was determined as an effective cut-off value across all cancer types to assess whether a nucleus is irregular. The EFC ratio showed cancer type-dependent correlation with tumor grades, with cancer cells in breast and head and neck possessing substantial nuclear irregularity, and colon, ovary, and fallopian tube cancer cells showing less irregularity than adjacent normal tissue. Overall, nuclear irregularity quantified using the EFC ratio has the potential as a powerful cancer diagnostic or prognostic marker. 


This research demonstrates the superiority of lamin stains as a marker of nuclear irregularity and the usefulness of the EFC ratio, along with nuclear area and aspect ratio, in quantifying nuclear contour irregularity. Furthermore, the relationships shown between these parameters and tumor grade strongly suggests the usefulness of these parameters and this method as a cancer diagnostic or prognostic tool.