Margaret M. Kelly, 91, matriarch of Philadelphia's fabled Kelly family and mother of the late Princess Grace of Monaco, died yesterday, after many years of ill health, at a convalescent home near Somers Point, N.J.

Mrs. Kelly, a former beauty queen, magazine "cover girl" and college athletic director, was the wife of a bricklayer - John B. Kelly Sr. - who became an Olympic rowing champion and a multimillionaire construction and brickwork magnate.

She had suffered several strokes in recent years. As a result, she never learned of the 1982 death of her daughter Grace Kelly, the film actress who became Princess Grace of Monaco, or the 1985 death of her son, John B. Kelly Jr., also an Olympic champion rower and president of the U.S. Olympic Committee.

Liz Kelly, Mrs. Kelly's granddaughter and a well-known Philadelphia restaurateur, said last night that her grandmother died about 8 a.m. yesterday. Her death was confirmed by the supervisor of the Linwood Convalescent Center.

In an introduction to the 1957 book That Kelly Family, Bill Corum wrote that "the story of the Kellys reads a bit like Merriwell out of Horatio Alger, almost fictional in its telling . . . the saga of an extraordinary clan."

Margaret Kelly was a strong force in and, many said, a major power behind the Kellys, a vigorous, achievement-oriented family that has been called Philadelphia's answer to the Kennedys. Like the Kennedys, too, the family had long been active in Democratic Party politics.

In an oft-told anecdote from 1975, Mrs. Kelly was said to have blocked her son's possible bid for mayor by calling then-Democratic Party chairman Peter J. Camiel and telling him not to slate her son to run against Mayor Frank L. Rizzo. "Kell's not suited to politics," she declared. "He can't say no. He's too good-natured a guy. He should stick to sports and business."

Her late husband, who died in 1960, was the youngest of 10 children of Irish immigrants who settled in Philadelphia. He began his career as a bricklayer and at his death in 1960 was the wealthy owner of a construction empire, having parlayed a $5,000 loan into a fortune.

Margaret Kelly was Margaret Majer of the Strawberry Mansion section of Philadelphia when, as a teenager, she met her future husband at a Philadelphia swimming pool in 1914.

Her photograph appeared on the covers of several national and Philadelphia magazines in 1919 and the early 1920s. She also was an accomplished swimmer and athlete and the first director of women's athletics at the University of Pennsylvania.

She married Kelly in 1924, after he had won several Olympic rowing championships. In 1920, in an often-recounted incident, he had been banned

from rowing in Britain's Diamond Sculls championships because he was a bricklayer - a social standing too common for Henley-on-Thames.

Princess Grace, who first became a queen in Hollywood and then, in a real- life Cinderella story, the princess of Monaco, died Sept. 14, 1982, of injuries she suffered in a car crash outside a small French town in mountainous country near Monaco.

Her brother died March 2, 1985, of a heart attack while jogging in Philadelphia after a morning of rowing on the Schuylkill, where he had spent countless hours honing his own Olympic-medal-winning skills. He had served two terms as city councilman-at-large and was active in the city's civic life.

Margaret Kelly was often described as a strong woman who had a major role in rearing the couple's children while her husband devoted much time to his sports and civic affairs.

In his 1977 book, Those Philadelphia Kellys - With a Touch of Grace, Arthur H. Lewis called her a "domineering force in the family" who began married life within the strong Kelly family as neither Irish nor Roman Catholic. Being

neither initially was a handicap for a Kelly, Lewis wrote.

Of German heritage, she later abandoned her own Protestant faith to become a convert to Catholicism and joined St. Bridget's Parish in East Falls.

She became known within the family as "Ma" Kelly, Lewis wrote.

John McCallum, in his book about the Kellys, That Kelly Family, wrote, ''Always she has stood loyally, staunchly in the shadow of her famous husband."

But Lewis wrote that her husband, a boxer in the Army before he won a battlefield commission directing an ambulance corps in France in World War I, was often difficult to get along with.

Lewis quoted Margaret as saying: "It's very difficult being married to a Kelly."

In addition to Grace Kelly, who was 52 when she died, and Jack Kelly, who died at 57, Margaret and Jack Sr. had two other children: Margaret R. Conlan and Lizanne K. LeVine, who have remained for the most part out of public life.

Liz Kelly said that funeral services were planned for Tuesday and that arrangements would be made by the McIlvaine Funeral Home in East Falls, the longtime home of the Kelly family. Mrs. Kelly moved from East Falls to Germantown in 1970.

Margaret M Kelly